-- Logs begin at Mon 2022-08-08 15:45:45 UTC, end at Mon 2022-08-08 17:16:05 UTC. -- Aug 08 16:30:26.414176 dv-cinder-852209-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 08 16:30:26.422466 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 08 16:30:26.426526 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 8 16:30:26 2022] *** Aug 08 16:30:26.426526 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 08 16:30:26.426526 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 08 16:30:26.426526 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: nodename: dv-cinder-852209-3-iscsi Aug 08 16:30:26.426526 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: machine: x86_64 Aug 08 16:30:26.426526 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: clock source: unix Aug 08 16:30:26.426526 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: pcre jit disabled Aug 08 16:30:26.426526 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: detected number of CPU cores: 4 Aug 08 16:30:26.426526 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: current working directory: / Aug 08 16:30:26.426526 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: detected binary path: /usr/bin/uwsgi-core Aug 08 16:30:26.426526 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: your processes number limit is 47680 Aug 08 16:30:26.426526 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: your memory page size is 4096 bytes Aug 08 16:30:26.426526 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: detected max file descriptor number: 2048 Aug 08 16:30:26.426526 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: lock engine: pthread robust mutexes Aug 08 16:30:26.426526 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: thunder lock: enabled Aug 08 16:30:26.426526 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 08 16:30:26.426526 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 08 16:30:26.468219 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: Python main interpreter initialized at 0x55c28c49e9c0 Aug 08 16:30:26.468219 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: python threads support enabled Aug 08 16:30:26.468219 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: your server socket listen backlog is limited to 100 connections Aug 08 16:30:26.468219 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: your mercy for graceful operations on workers is 90 seconds Aug 08 16:30:26.468983 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: mapped 403077 bytes (393 KB) for 2 cores Aug 08 16:30:26.468983 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: *** Operational MODE: preforking *** Aug 08 16:30:26.469210 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: *** uWSGI is running in multiple interpreter mode *** Aug 08 16:30:26.469210 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: spawned uWSGI master process (pid: 81978) Aug 08 16:30:26.469425 dv-cinder-852209-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 08 16:30:26.470626 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: spawned uWSGI worker 1 (pid: 81998, cores: 1) Aug 08 16:30:26.470626 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: spawned uWSGI worker 2 (pid: 81999, cores: 1) Aug 08 16:30:26.470626 dv-cinder-852209-3-iscsi devstack@keystone.service[81978]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 08 16:30:32.257826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 16:30:32.257826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: from cryptography.utils import int_from_bytes Aug 08 16:30:32.258535 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 16:30:32.258535 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: from cryptography.utils import int_from_bytes Aug 08 16:30:32.311538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 16:30:32.311783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 16:30:32.311907 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] command line args: [] {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 16:30:32.312155 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 16:30:32.312500 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ================================================================================ {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 16:30:32.312956 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] admin_token = **** {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.313400 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] config_dir = [] {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.313747 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.314078 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] config_source = [] {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.314406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] debug = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.314811 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.315272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.315591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.316219 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.316656 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.317079 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] list_limit = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.317407 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] log_config_append = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.317733 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.318057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] log_dir = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.318378 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] log_file = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.318865 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.319146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.319447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.319746 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.319930 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.320114 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.320405 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.320734 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.320980 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.321300 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.321623 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.321788 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.322065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.322344 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.322818 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.323399 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.323704 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] publish_errors = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.324840 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.324840 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.324840 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.325412 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.325412 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.325913 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] standard_threads = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.326267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.326670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.326670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.327358 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] use_journal = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.327358 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] use_json = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.327358 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] use_stderr = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.327358 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] use_syslog = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.327944 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.327944 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.328163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.328581 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.328807 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.329020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.329288 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.329621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.329941 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.330294 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.330804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.331159 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.331418 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.331862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.332111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.332369 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.332636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.332938 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.333139 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.333399 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.333653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.333896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.334159 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.334435 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.334728 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.334979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.335217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.335454 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.335693 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.336052 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.336197 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.336485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.336719 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.336964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.337217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.337472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.337713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.338131 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.338360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.338785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.339047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.339277 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.339506 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.339764 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.340060 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.340431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.340689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.340931 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.341302 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.341437 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.341672 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.341897 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.342156 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.342378 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.342678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.342890 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.343088 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.343653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.343876 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.344078 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.344289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.344618 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.344868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.345094 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.345610 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.345610 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.346031 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.346031 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.346415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.346663 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.346874 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.347070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.347267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.347510 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.347748 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.347988 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.348242 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.348484 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.348726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.348994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.349211 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.349461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.349720 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.349945 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.350159 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.350472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.350792 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.351036 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.351244 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.351510 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.351709 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.351910 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.352125 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.352342 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.352547 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.352795 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.353033 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.353275 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.353520 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.353841 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.354051 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.354453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.354453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.354895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.355093 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.355781 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] auth.external = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.355976 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.356276 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.356577 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.356901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] auth.password = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.357141 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] auth.token = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.357371 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.357615 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.357886 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.358099 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.358318 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.358741 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.359073 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.359073 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] credential.caching = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.359475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.359679 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.359909 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.360329 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.360329 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.361339 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.361612 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.362006 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.362006 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.362623 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.362623 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.363054 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.363054 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.363451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.363451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.364006 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] federation.caching = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.364006 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.366448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.366448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.366448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.366448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.366448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.366448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.366448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.366448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.366448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.368321 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.368321 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] identity.caching = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.368321 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.368321 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.368321 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.368321 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.368321 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.368321 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.368321 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.370049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.370049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.370049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.370049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.370049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.370049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.370049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.370049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.372245 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.372245 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.372245 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.372245 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.372245 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.372245 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.372245 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.373605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.373605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.373605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.373605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.373605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.373605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.374779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.374779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.374779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.374779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.374779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.375755 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.375755 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.375755 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.375755 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.376636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.376636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.376636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.376636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.377459 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.377459 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.377459 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.378045 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.378045 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.378045 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.378667 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.378667 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.379084 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.379084 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.user = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.379478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.379478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.379881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.379881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.380282 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.380488 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.380692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.380896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.381096 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.381414 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.381414 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.381882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.381882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.382317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.382540 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.382772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.382989 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.383221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.383426 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.383635 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.383837 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.384046 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.384249 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.384463 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.384788 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.385022 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.385233 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.385440 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.385647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.385867 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.386496 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.386496 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.387012 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.387012 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.387700 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.387700 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.387700 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] resource.caching = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.388295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.388295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.388295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.388891 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.388891 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.389301 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.389301 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.389677 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.389677 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.390092 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] role.caching = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.390312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] role.driver = sql {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.390520 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.390753 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.390958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.391176 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.391375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.391612 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.391819 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.392101 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.392309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.392515 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.392728 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.393094 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.393320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.393526 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.393900 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.394119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.394376 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.394657 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.395069 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.395330 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.395775 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.396039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.396329 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.396597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.396887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.397104 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.397307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.397542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.397805 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.398060 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.398285 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.398512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.398808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.399018 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] token.caching = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.399229 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.399483 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.399759 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.399957 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.400465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.400465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.400859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.401070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.401292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.401500 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.401710 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.402127 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.402361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.402649 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.402887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.403298 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.403521 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.403732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] database.connection = **** {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.403957 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.404193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.404513 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.404742 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.405975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.406219 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.406548 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.406831 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.407043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.407341 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.407551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.407763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.408043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.408756 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.409461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.409703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.409957 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.410196 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.410362 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81999) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 16:30:32.467479 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.467793 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.467912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.480069 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.514134 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.514296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.571684 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.625547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 16:30:32.625547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: from cryptography.utils import int_from_bytes Aug 08 16:30:32.626210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 16:30:32.626210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: from cryptography.utils import int_from_bytes Aug 08 16:30:32.626782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.627022 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.675313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 16:30:32.675521 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 16:30:32.675682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] command line args: [] {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 16:30:32.675954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 16:30:32.676128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ================================================================================ {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 16:30:32.676383 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] admin_token = **** {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.676624 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] config_dir = [] {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.676808 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.676982 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] config_source = [] {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.677158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] debug = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.677368 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.677606 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.677804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.678060 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.678272 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.678500 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] list_limit = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.678732 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] log_config_append = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.678870 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.679036 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] log_dir = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.679202 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] log_file = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.679437 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.679683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.679927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.680173 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.680348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.680512 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.680749 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.680990 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.681227 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.681452 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.681686 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.681914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.682159 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.682399 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.682715 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.683199 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.683885 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] publish_errors = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.684117 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.684344 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.684511 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.684755 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.684980 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.685217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] standard_threads = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.685440 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.685682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.685856 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.686029 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] use_journal = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.686202 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] use_json = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.686376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] use_stderr = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.686551 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] use_syslog = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.686798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:30:32.687054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.687290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.687547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.687795 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.688038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.688273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.688517 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.688750 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.688997 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.689241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.689488 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.689727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.689949 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.690182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.690429 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.690925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.691166 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.691421 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.691673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.691920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.692154 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.692405 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.692667 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.692918 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.693163 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.693392 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.693619 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.693845 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.694081 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.694350 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.694630 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.694876 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.695106 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.695329 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.695518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.695751 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.696000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.696228 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.696463 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.696696 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.696924 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.697152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.697378 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.697611 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.697841 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.698086 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.698318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.698552 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.699763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.699763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.699963 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.700039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.700185 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.700289 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.700416 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.700519 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.700638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.700720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.700837 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.700924 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.701054 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.701145 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.701345 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.701575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.701801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.702055 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.702285 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.702530 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.703705 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.703705 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.703705 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.703705 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.704693 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.704942 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.705205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.705450 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.705696 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.705922 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.706162 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.706392 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.706828 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.707078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.707319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.707567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.707806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.708041 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.708277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.708515 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.708749 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.708983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.709245 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.709484 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.709723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.709958 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.710199 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.710433 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.710841 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.711079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.711308 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.711539 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.712014 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.712014 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.712213 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.712426 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.712664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.712912 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.713150 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.713395 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.713652 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.713892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.714122 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] auth.external = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.714354 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.714881 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.715001 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.715135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] auth.password = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.715359 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] auth.token = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.715598 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.716028 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.716205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.716434 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.716678 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.716928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.717157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.717390 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] credential.caching = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.717636 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.717883 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.718129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.718644 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.718889 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.719145 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.719684 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.719684 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.719844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.720094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.720387 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.720626 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.720887 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.721126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.721380 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.721618 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] federation.caching = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.721849 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.722093 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.722339 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.722691 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.722835 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.723073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.723330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.723563 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.723816 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.724051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.724298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.724534 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] identity.caching = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.724788 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.725146 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.725287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.725471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.725718 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.725947 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.726183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.726454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.726540 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.726922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.727016 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.727161 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.727272 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.727369 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.727449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.727589 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.727679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.727801 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.727917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.727999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.728166 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.728417 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.728672 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.728919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.729185 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.729428 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.729662 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.729912 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.730185 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.730622 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.731005 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.731221 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.731460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.731711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.731944 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.732186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.732430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.732663 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.732907 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.733148 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.733376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.733612 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.733840 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.734082 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.734310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.734555 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.734812 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.735049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.735299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.735548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.735803 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.736007 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.736253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.736572 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.736735 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.736974 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.737207 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.737435 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.user = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.737677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.737938 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.738167 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.738466 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.738724 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.738931 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.739163 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.739392 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.739626 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.739867 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.741173 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.741173 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.741173 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.741173 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.741449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.741520 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.741520 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.741520 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.741675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.741927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.742170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.742394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.742652 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.742888 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.743132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.743378 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.743997 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.743997 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.744868 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.744868 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.744868 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.744868 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.745335 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.745335 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.745552 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.745670 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.746407 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.746734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] resource.caching = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.746957 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.747209 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.747444 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.748108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.748108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.748333 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.748413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.748617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.748864 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.749096 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] role.caching = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.749342 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] role.driver = sql {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.749672 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.749890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.750121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.750358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.750560 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.750849 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.752138 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.752138 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.752138 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.752138 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.752138 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.752552 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.752552 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.752680 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.753126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.753365 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.753619 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.753879 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.754128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.754382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.754754 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.755088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.755337 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.755580 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.755812 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.756041 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.756126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.756270 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.756349 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.756494 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.756575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.756717 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.756798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.756936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.757018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.757161 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.757238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.757354 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81999) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.757435 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.757665 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.757898 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] token.caching = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.758134 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.758382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.758649 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.758914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.759160 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.759397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.759699 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.759894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.760145 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.760380 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.760617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.760856 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.761098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.761381 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.761824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.762090 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.762910 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.763179 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] database.connection = **** {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.763436 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.763704 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.763925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.764213 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.764441 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.764654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.764872 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.765111 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.765347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.765570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.765784 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.766018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.766267 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.766502 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.766773 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.767012 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.767264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.767503 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:30:32.767687 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81998) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 16:30:32.776253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.776442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 16:30:32.776797 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.777565 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 16:30:32.778018 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.778378 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.778834 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.778980 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.790825 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.791121 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.791393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.803331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.803467 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.803467 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.803467 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.803646 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81999) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:32.803646 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81999) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:32.803646 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.803646 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.803873 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.804070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.804334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.804606 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.804770 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.804905 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.805152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.805427 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.805698 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.823546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.823872 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.830175 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.831756 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81999) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:32.832044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.836786 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81999) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:32.837085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.837370 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.845520 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81999) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:32.845937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.850357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81999) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:32.850686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.850982 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.851274 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.851729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.852025 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.852320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.852589 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.863136 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.869853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.870179 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.874364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81999) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:32.877635 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.877977 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.882477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81999) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:32.885854 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81999) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:32.889281 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.889584 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.889876 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.890174 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.890461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.890790 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.900679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.900956 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.910055 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81999) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:32.910360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.910652 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.910926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.919014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.920566 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.920848 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.923828 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81999) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:32.924116 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.924576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.925028 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.925324 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.937376 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81999) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:32.937686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.937969 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.938249 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.938537 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.938885 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.939163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.945070 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.945297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.945504 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.945722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.945934 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.946143 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.957131 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81999) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:32.959833 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.960060 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.960222 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81999) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:32.960328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.960554 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.960700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.960914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.961124 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.963249 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.965788 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.967344 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81999) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:32.967680 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.967967 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.969872 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.973069 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.973298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.973524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.973746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.973960 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.974170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.974381 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81998) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:30:32.974891 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.976553 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81999) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:32.979797 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.980112 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.980946 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.981221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.984977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.985183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 16:30:32.985701 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.985945 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 16:30:32.986235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.986464 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.986711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.986970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.989797 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81999) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:32.990107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.990400 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.998121 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81999) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:32.998282 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.998451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.998858 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.998962 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.999179 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.999284 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.999471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.999556 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.999731 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:32.999835 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.000019 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.000362 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.000641 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.000919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.001195 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.001483 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.001755 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81999) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.006904 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81998) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:33.010037 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81998) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:33.010370 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.010700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.010944 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.011179 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.011959 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.011959 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.012565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.012565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.012917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.013019 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.037077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.037570 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55c28c49e9c0 pid: 81999 (default app) Aug 08 16:30:33.039279 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81998) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:33.039717 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.041089 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462260ec-a4a9-4f63-bb4c-b5ea11dba340 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:33.041377 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462260ec-a4a9-4f63-bb4c-b5ea11dba340 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:33.041645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462260ec-a4a9-4f63-bb4c-b5ea11dba340 None None] PATH_INFO: `/v3` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:33.046775 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Mon Aug 8 16:30:33 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:30:33.047610 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81998) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:33.048095 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.048452 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.060780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81998) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:33.061237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.067695 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81998) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:33.068113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.068470 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.069613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.069613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.069877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.070617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.070617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.088265 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.088564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.093546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81998) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:33.096835 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.097135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.101464 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81998) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:33.104812 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81998) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:33.108111 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.108410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.108693 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.108972 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.109406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.109695 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.126025 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81998) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:33.126414 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.126880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.127473 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.139117 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.141765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.142198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.146885 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81998) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:33.147374 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.147851 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.148319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.148790 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.163753 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81998) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:33.164055 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.164337 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.164620 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.164902 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.165178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.165462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.185647 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81998) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:33.188750 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81998) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:33.192810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.194683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.196341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81998) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:33.196778 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.197223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.204392 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.206114 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81998) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:33.209431 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.209893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.210395 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.211104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.220631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81998) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:33.221292 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.221729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.231077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81998) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:30:33.231385 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.231675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.231955 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.232232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.232510 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.232788 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.233062 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.233351 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.237367 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.237687 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.237966 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.238382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.238695 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81998) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:30:33.274317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55c28c49e9c0 pid: 81998 (default app) Aug 08 16:30:42.122774 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c89a57-caaf-4767-a8f3-2d9aac79439e None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:42.123352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c89a57-caaf-4767-a8f3-2d9aac79439e None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:42.123352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c89a57-caaf-4767-a8f3-2d9aac79439e None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:42.124595 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 2/2] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:30:42.132313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2765c-6b83-48d7-aa91-509d2e76258b None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:42.132818 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2765c-6b83-48d7-aa91-509d2e76258b None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:42.132940 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2765c-6b83-48d7-aa91-509d2e76258b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:42.206331 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: INFO dbcounter [None req-07f2765c-6b83-48d7-aa91-509d2e76258b None None] Registered counter for database keystone Aug 08 16:30:42.226744 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG oslo_db.sqlalchemy.engines [None req-07f2765c-6b83-48d7-aa91-509d2e76258b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81998) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 16:30:42.233571 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writer thread running {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 16:30:42.421679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG passlib.handlers.bcrypt [None req-07f2765c-6b83-48d7-aa91-509d2e76258b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81998) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 08 16:30:42.422092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG passlib.handlers.bcrypt [None req-07f2765c-6b83-48d7-aa91-509d2e76258b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81998) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 08 16:30:42.479704 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-07f2765c-6b83-48d7-aa91-509d2e76258b None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:30:42.481975 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-07f2765c-6b83-48d7-aa91-509d2e76258b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:30:42.631163 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1/3] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:30:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 08 16:30:42.787543 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c89a57-caaf-4767-a8f3-2d9aac79439e None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:42.868709 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: INFO dbcounter [None req-37c89a57-caaf-4767-a8f3-2d9aac79439e None None] Registered counter for database keystone Aug 08 16:30:42.892131 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG oslo_db.sqlalchemy.engines [None req-37c89a57-caaf-4767-a8f3-2d9aac79439e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81999) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 16:30:42.898853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writer thread running {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 16:30:43.023350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c89a57-caaf-4767-a8f3-2d9aac79439e None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:43.028051 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c5c5a-0f05-4463-9024-beab7d978243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:43.028758 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c5c5a-0f05-4463-9024-beab7d978243 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:43.029044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c5c5a-0f05-4463-9024-beab7d978243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:43.029315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c5c5a-0f05-4463-9024-beab7d978243 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:43.040878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c5c5a-0f05-4463-9024-beab7d978243 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:43.093486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4b3c5c5a-0f05-4463-9024-beab7d978243 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 08 16:30:43.093828 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c5c5a-0f05-4463-9024-beab7d978243 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:43.104524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 3/4] 10.222.0.13 () {58 vars in 1324 bytes} [Mon Aug 8 16:30:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 16:30:43.111229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2765c-6b83-48d7-aa91-509d2e76258b None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:43.131321 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2765c-6b83-48d7-aa91-509d2e76258b None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:43.137611 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cad80f-27c3-41de-849c-a37f7cd1edee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:43.138839 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cad80f-27c3-41de-849c-a37f7cd1edee admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:43.139121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cad80f-27c3-41de-849c-a37f7cd1edee admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:43.139386 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cad80f-27c3-41de-849c-a37f7cd1edee admin admin] PATH_INFO: `/v3/projects/3a8d1eb28476411285e5ac5c0ab9ab53` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:43.141063 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cad80f-27c3-41de-849c-a37f7cd1edee admin admin] RBAC: Authorizing `identity:get_project(project_id=3a8d1eb28476411285e5ac5c0ab9ab53)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:43.203301 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: WARNING keystone.common.rbac_enforcer.enforcer [None req-52cad80f-27c3-41de-849c-a37f7cd1edee admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 08 16:30:43.207062 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cad80f-27c3-41de-849c-a37f7cd1edee admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:43.208952 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 2/5] 10.222.0.13 () {56 vars in 1207 bytes} [Mon Aug 8 16:30:43 2022] GET /identity/v3/projects/3a8d1eb28476411285e5ac5c0ab9ab53 => generated 344 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:30:45.229157 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5805a-d5e7-4f9d-b455-164955c7c331 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:45.229670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5805a-d5e7-4f9d-b455-164955c7c331 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:45.229670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5805a-d5e7-4f9d-b455-164955c7c331 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:45.234805 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 4/6] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:30:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:30:45.240370 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cd47b-07e6-4ce5-bb45-d6ee7182e3fe None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:45.240821 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cd47b-07e6-4ce5-bb45-d6ee7182e3fe None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:45.240998 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cd47b-07e6-4ce5-bb45-d6ee7182e3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:45.262716 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-225cd47b-07e6-4ce5-bb45-d6ee7182e3fe None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:30:45.264277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-225cd47b-07e6-4ce5-bb45-d6ee7182e3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:30:45.272144 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 3/7] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:30:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 08 16:30:45.295832 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5805a-d5e7-4f9d-b455-164955c7c331 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:45.308933 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5805a-d5e7-4f9d-b455-164955c7c331 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:45.313293 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16dddbd-dd00-4e43-82a2-4e9254ed89a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:45.313894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16dddbd-dd00-4e43-82a2-4e9254ed89a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:45.314165 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16dddbd-dd00-4e43-82a2-4e9254ed89a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:45.314433 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16dddbd-dd00-4e43-82a2-4e9254ed89a8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:45.322370 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16dddbd-dd00-4e43-82a2-4e9254ed89a8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:45.324545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16dddbd-dd00-4e43-82a2-4e9254ed89a8 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:45.333350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 5/8] 10.222.0.13 () {58 vars in 1324 bytes} [Mon Aug 8 16:30:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 16:30:45.339337 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225cd47b-07e6-4ce5-bb45-d6ee7182e3fe None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:45.349019 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225cd47b-07e6-4ce5-bb45-d6ee7182e3fe None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:45.353445 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf7420-1ec3-49b6-a1f0-c7e2571e9998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:45.354073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf7420-1ec3-49b6-a1f0-c7e2571e9998 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:45.354350 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf7420-1ec3-49b6-a1f0-c7e2571e9998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:45.354647 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf7420-1ec3-49b6-a1f0-c7e2571e9998 admin admin] PATH_INFO: `/v3/users/b308731f39014d7098d27d547158b386` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:45.356588 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf7420-1ec3-49b6-a1f0-c7e2571e9998 admin admin] RBAC: Authorizing `identity:get_user(user_id=b308731f39014d7098d27d547158b386)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:45.358420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf7420-1ec3-49b6-a1f0-c7e2571e9998 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:45.359987 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 4/9] 10.222.0.13 () {56 vars in 1198 bytes} [Mon Aug 8 16:30:45 2022] GET /identity/v3/users/b308731f39014d7098d27d547158b386 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:30:47.480912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea0c61-108a-44f8-8f95-f53061075c1b None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:47.481484 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea0c61-108a-44f8-8f95-f53061075c1b None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:47.481484 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea0c61-108a-44f8-8f95-f53061075c1b None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:47.482813 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 6/10] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:30:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:30:47.488415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d8c79-6abe-4091-b752-ba7c1d915fd2 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:47.488940 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d8c79-6abe-4091-b752-ba7c1d915fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:47.488940 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d8c79-6abe-4091-b752-ba7c1d915fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:47.501036 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c86443-4f44-443d-9212-33952d01fb17 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:47.501378 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c86443-4f44-443d-9212-33952d01fb17 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:47.501825 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c86443-4f44-443d-9212-33952d01fb17 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:47.504148 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 7/11] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:30:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:30:47.509343 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-182d8c79-6abe-4091-b752-ba7c1d915fd2 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:30:47.510697 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-182d8c79-6abe-4091-b752-ba7c1d915fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:30:47.512703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be80a36-dee2-4f1a-9782-fc4054928306 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:47.513106 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be80a36-dee2-4f1a-9782-fc4054928306 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:47.513505 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be80a36-dee2-4f1a-9782-fc4054928306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:47.561398 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG passlib.handlers.bcrypt [None req-9be80a36-dee2-4f1a-9782-fc4054928306 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81999) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 08 16:30:47.561829 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG passlib.handlers.bcrypt [None req-9be80a36-dee2-4f1a-9782-fc4054928306 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81999) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 08 16:30:47.595498 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 5/12] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:30:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:30:47.610243 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-9be80a36-dee2-4f1a-9782-fc4054928306 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:30:47.611995 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-9be80a36-dee2-4f1a-9782-fc4054928306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:30:47.619132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d8c79-6abe-4091-b752-ba7c1d915fd2 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:47.644069 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:47.723195 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 8/13] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:30:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:30:47.726781 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d8c79-6abe-4091-b752-ba7c1d915fd2 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:47.757631 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be80a36-dee2-4f1a-9782-fc4054928306 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:47.761554 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a63e2-3e87-4b8d-afdd-cd8ae603cef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:47.762203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a63e2-3e87-4b8d-afdd-cd8ae603cef1 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:47.762479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a63e2-3e87-4b8d-afdd-cd8ae603cef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:47.762774 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a63e2-3e87-4b8d-afdd-cd8ae603cef1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:47.774797 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a63e2-3e87-4b8d-afdd-cd8ae603cef1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:30:47.783786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-790a63e2-3e87-4b8d-afdd-cd8ae603cef1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:30:47.783786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:30:47.783786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:30:47.783786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:30:47.783786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:30:47.783786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:30:47.783786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:30:47.783786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:30:47.783786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:30:47.783786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:30:47.783786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:30:47.783786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:30:47.783786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:30:47.783786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:30:47.783786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:30:47.783786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:30:47.785575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:30:47.785575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:30:47.785575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:30:47.785575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:30:47.785575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:30:47.785575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:30:47.785575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:30:47.785575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:30:47.785575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:30:47.785575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:30:47.785575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:30:47.785575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:30:47.785575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:30:47.785575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:30:47.785575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:30:47.785575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:30:47.787133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:30:47.787133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:30:47.787133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:30:47.787133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:30:47.787133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:30:47.787133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:30:47.787133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:30:47.787133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:30:47.787133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:30:47.787133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:30:47.787133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:30:47.787133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:30:47.787133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 6/14] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:30:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:30:47.791191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a63e2-3e87-4b8d-afdd-cd8ae603cef1 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:47.868542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be80a36-dee2-4f1a-9782-fc4054928306 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:47.871637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:47.878699 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:47.896326 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a63e2-3e87-4b8d-afdd-cd8ae603cef1 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:47.907348 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f4786-d883-437c-958b-4f18da28094f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:47.908221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f4786-d883-437c-958b-4f18da28094f None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:47.908505 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f4786-d883-437c-958b-4f18da28094f None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:47.908772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f4786-d883-437c-958b-4f18da28094f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:47.932510 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3e48e-e866-4993-a8db-e68081efdf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:47.933149 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3e48e-e866-4993-a8db-e68081efdf85 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:47.933427 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3e48e-e866-4993-a8db-e68081efdf85 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:47.933695 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3e48e-e866-4993-a8db-e68081efdf85 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:47.934366 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3e48e-e866-4993-a8db-e68081efdf85 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:30:47.934713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3e48e-e866-4993-a8db-e68081efdf85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:47.936244 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3e48e-e866-4993-a8db-e68081efdf85 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:47.948931 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 7/15] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:30:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:30:47.954560 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3e48e-e866-4993-a8db-e68081efdf85 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:47.978690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f4786-d883-437c-958b-4f18da28094f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:47.981166 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f4786-d883-437c-958b-4f18da28094f None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:48.089020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 9/16] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:30:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 16:30:48.095849 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f4786-d883-437c-958b-4f18da28094f None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:48.099112 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:48.166504 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3e48e-e866-4993-a8db-e68081efdf85 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:48.202734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff1562-050a-4d86-915c-08a4d756ba12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:48.203372 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff1562-050a-4d86-915c-08a4d756ba12 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:48.203648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff1562-050a-4d86-915c-08a4d756ba12 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:48.203914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff1562-050a-4d86-915c-08a4d756ba12 None admin] PATH_INFO: `/v3/users/b308731f39014d7098d27d547158b386` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:48.204133 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f4786-d883-437c-958b-4f18da28094f None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:48.205052 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff1562-050a-4d86-915c-08a4d756ba12 None admin] RBAC: Authorizing `identity:get_user(user_id=b308731f39014d7098d27d547158b386)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:48.206511 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff1562-050a-4d86-915c-08a4d756ba12 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:48.208178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 8/17] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:30:47 2022] GET /identity/v3/users/b308731f39014d7098d27d547158b386 => generated 247 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:30:48.213585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff1562-050a-4d86-915c-08a4d756ba12 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:48.229389 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:48.240495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea463e9-5588-4633-9d85-b630f39fb078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:48.241360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea463e9-5588-4633-9d85-b630f39fb078 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:48.241639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea463e9-5588-4633-9d85-b630f39fb078 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:48.246913 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea463e9-5588-4633-9d85-b630f39fb078 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:48.261436 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea463e9-5588-4633-9d85-b630f39fb078 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:48.262965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea463e9-5588-4633-9d85-b630f39fb078 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:48.282534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-0ea463e9-5588-4633-9d85-b630f39fb078 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:30:48.282534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:30:48.282534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:30:48.282534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:30:48.282534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:30:48.282534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:30:48.282534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:30:48.282534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:30:48.282534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:30:48.282534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:30:48.282534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:30:48.282534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:30:48.282534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:30:48.282534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:30:48.282534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:30:48.282534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:30:48.283669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:30:48.283669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:30:48.283669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:30:48.283669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:30:48.283669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:30:48.283669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:30:48.283669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:30:48.283669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:30:48.283669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:30:48.283669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:30:48.283669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:30:48.283669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:30:48.283669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:30:48.283669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:30:48.283669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:30:48.283669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:30:48.283669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:30:48.284603 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:30:48.284603 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:30:48.284603 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:30:48.284603 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 10/18] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:30:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:30:48.289789 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea463e9-5588-4633-9d85-b630f39fb078 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:48.319855 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff1562-050a-4d86-915c-08a4d756ba12 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:48.348099 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:48.355739 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d1ec1-bf4f-4c74-a26c-1857c99da74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:48.356379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d1ec1-bf4f-4c74-a26c-1857c99da74a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:48.356659 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d1ec1-bf4f-4c74-a26c-1857c99da74a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:48.367075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d1ec1-bf4f-4c74-a26c-1857c99da74a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:48.367075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993d1ec1-bf4f-4c74-a26c-1857c99da74a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:48.367075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993d1ec1-bf4f-4c74-a26c-1857c99da74a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:48.367075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 9/19] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:30:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:30:48.367075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d1ec1-bf4f-4c74-a26c-1857c99da74a None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:48.393341 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea463e9-5588-4633-9d85-b630f39fb078 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:48.428529 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991519f-34a2-4f2c-9da9-959022cdbe04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:48.429151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991519f-34a2-4f2c-9da9-959022cdbe04 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:48.429422 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991519f-34a2-4f2c-9da9-959022cdbe04 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:48.429702 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991519f-34a2-4f2c-9da9-959022cdbe04 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:48.430362 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991519f-34a2-4f2c-9da9-959022cdbe04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:30:48.430692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991519f-34a2-4f2c-9da9-959022cdbe04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:48.432124 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991519f-34a2-4f2c-9da9-959022cdbe04 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:48.450681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:48.451310 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 11/20] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:30:48.475512 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d1ec1-bf4f-4c74-a26c-1857c99da74a None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:48.515013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59328a94-1f85-4e90-8c5a-84e36f3db546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:48.515674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59328a94-1f85-4e90-8c5a-84e36f3db546 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:48.515889 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59328a94-1f85-4e90-8c5a-84e36f3db546 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:48.516149 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59328a94-1f85-4e90-8c5a-84e36f3db546 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:48.516882 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59328a94-1f85-4e90-8c5a-84e36f3db546 None admin] RBAC: Adding query filter params (role.id=e578ef831127463d9b48cc89fe20329c, scope.domain.id=default, user.id=b308731f39014d7098d27d547158b386) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:30:48.517182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59328a94-1f85-4e90-8c5a-84e36f3db546 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:48.519004 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59328a94-1f85-4e90-8c5a-84e36f3db546 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:48.538603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 10/21] 10.222.0.13 () {56 vars in 1322 bytes} [Mon Aug 8 16:30:48 2022] GET /identity/v3/role_assignments?user.id=b308731f39014d7098d27d547158b386&scope.domain.id=default&role.id=e578ef831127463d9b48cc89fe20329c => generated 233 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:30:50.947807 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63641da-f9ff-41d9-aecb-0e4f28b1dcce None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:50.948358 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63641da-f9ff-41d9-aecb-0e4f28b1dcce None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:50.948358 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63641da-f9ff-41d9-aecb-0e4f28b1dcce None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:50.949828 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 12/22] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:30:50.955622 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98eb4ee-3988-49d9-90fd-f8ba416af676 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:50.956104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98eb4ee-3988-49d9-90fd-f8ba416af676 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:50.956193 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98eb4ee-3988-49d9-90fd-f8ba416af676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:50.986231 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-c98eb4ee-3988-49d9-90fd-f8ba416af676 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:30:50.987471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-c98eb4ee-3988-49d9-90fd-f8ba416af676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:30:51.077612 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 11/23] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:30:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:30:51.107813 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63641da-f9ff-41d9-aecb-0e4f28b1dcce None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:51.171431 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183dfff-a4ef-462b-a98b-07eb30303866 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:51.172066 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183dfff-a4ef-462b-a98b-07eb30303866 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:51.172582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183dfff-a4ef-462b-a98b-07eb30303866 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:51.174158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 12/24] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:30:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:30:51.179377 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68bb11-9062-4d34-b511-a09ace753be0 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:51.179980 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68bb11-9062-4d34-b511-a09ace753be0 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:51.180464 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68bb11-9062-4d34-b511-a09ace753be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:51.211610 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-5c68bb11-9062-4d34-b511-a09ace753be0 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:30:51.213161 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-5c68bb11-9062-4d34-b511-a09ace753be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:30:51.240272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63641da-f9ff-41d9-aecb-0e4f28b1dcce None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:51.267196 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:51.298599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b81b9-832f-44d7-839e-b57ef0a580c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:51.298599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b81b9-832f-44d7-839e-b57ef0a580c9 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:51.298599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b81b9-832f-44d7-839e-b57ef0a580c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:51.298599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b81b9-832f-44d7-839e-b57ef0a580c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:51.314410 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b81b9-832f-44d7-839e-b57ef0a580c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:30:51.321057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-e07b81b9-832f-44d7-839e-b57ef0a580c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:30:51.321057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:30:51.321057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:30:51.321057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:30:51.321057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:30:51.321057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:30:51.321057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:30:51.321057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:30:51.321057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:30:51.321057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:30:51.321057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:30:51.321057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:30:51.321057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:30:51.321057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:30:51.321057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:30:51.321057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:30:51.321919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:30:51.321919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:30:51.321919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:30:51.321919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:30:51.321919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:30:51.321919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:30:51.321919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:30:51.321919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:30:51.321919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:30:51.321919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:30:51.321919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:30:51.321919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:30:51.321919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:30:51.321919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:30:51.321919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:30:51.321919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:30:51.322722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:30:51.322722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:30:51.322722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:30:51.322722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:30:51.322722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:30:51.322722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:30:51.322722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:30:51.322722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:30:51.322722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:30:51.322722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:30:51.322722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:30:51.322722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:30:51.328609 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 13/25] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:30:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:30:51.334634 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b81b9-832f-44d7-839e-b57ef0a580c9 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:51.367951 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 13/26] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:30:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:30:51.369255 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074c16e-be20-486d-bea3-995c636ae5f5 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:51.369543 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074c16e-be20-486d-bea3-995c636ae5f5 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:51.369818 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074c16e-be20-486d-bea3-995c636ae5f5 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:51.371919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 14/27] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:30:51.387645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f37b9-b016-4748-986a-36b22210e67e None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:51.387946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f37b9-b016-4748-986a-36b22210e67e None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:51.388193 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f37b9-b016-4748-986a-36b22210e67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:51.412238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:51.439640 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-cf4f37b9-b016-4748-986a-36b22210e67e None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:30:51.441164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-cf4f37b9-b016-4748-986a-36b22210e67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:30:51.468995 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b81b9-832f-44d7-839e-b57ef0a580c9 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:51.517906 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 15/28] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:30:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:30:51.519092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f37b9-b016-4748-986a-36b22210e67e None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:51.629688 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f37b9-b016-4748-986a-36b22210e67e None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:51.643567 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d36a6-cce5-4aba-9d03-36476be020ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:51.644364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d36a6-cce5-4aba-9d03-36476be020ea None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:51.644733 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d36a6-cce5-4aba-9d03-36476be020ea None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:51.645091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d36a6-cce5-4aba-9d03-36476be020ea None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:51.646090 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8d36a6-cce5-4aba-9d03-36476be020ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:30:51.646498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8d36a6-cce5-4aba-9d03-36476be020ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:51.648561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8d36a6-cce5-4aba-9d03-36476be020ea None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:51.668174 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 14/29] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:30:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:30:51.669809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8184dc2f-4796-468a-98fc-8e2dee1be835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:51.669953 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c8605-7b3a-4f25-8f6d-fc88fb8af168 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:51.670181 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c8605-7b3a-4f25-8f6d-fc88fb8af168 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:51.670535 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c8605-7b3a-4f25-8f6d-fc88fb8af168 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:51.671193 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184dc2f-4796-468a-98fc-8e2dee1be835 None admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:51.671674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184dc2f-4796-468a-98fc-8e2dee1be835 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:51.672157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184dc2f-4796-468a-98fc-8e2dee1be835 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:51.673018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8184dc2f-4796-468a-98fc-8e2dee1be835 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:51.674666 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8184dc2f-4796-468a-98fc-8e2dee1be835 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:51.678341 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 15/30] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:30:51 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:30:51.683819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:51.683984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c8605-7b3a-4f25-8f6d-fc88fb8af168 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:51.730230 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:51.807462 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c8605-7b3a-4f25-8f6d-fc88fb8af168 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:51.846024 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36524d2e-8316-4b70-a475-50c268e9483d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:51.847044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36524d2e-8316-4b70-a475-50c268e9483d None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:51.847546 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36524d2e-8316-4b70-a475-50c268e9483d None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:51.848042 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36524d2e-8316-4b70-a475-50c268e9483d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:51.859731 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36524d2e-8316-4b70-a475-50c268e9483d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:51.861543 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36524d2e-8316-4b70-a475-50c268e9483d None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:51.877868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-36524d2e-8316-4b70-a475-50c268e9483d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:30:51.877868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:30:51.877868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:30:51.877868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:30:51.877868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:30:51.877868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:30:51.877868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:30:51.877868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:30:51.877868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:30:51.877868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:30:51.877868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:30:51.877868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:30:51.877868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:30:51.877868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:30:51.877868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:30:51.877868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:30:51.878830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:30:51.878830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:30:51.878830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:30:51.878830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:30:51.878830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:30:51.878830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:30:51.878830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:30:51.878830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:30:51.878830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:30:51.878830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:30:51.878830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:30:51.878830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:30:51.878830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:30:51.878830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:30:51.878830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:30:51.878830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:30:51.878830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:30:51.879678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:30:51.879678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:30:51.879678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:30:51.881553 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 16/31] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:30:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:30:51.882596 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36524d2e-8316-4b70-a475-50c268e9483d None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:51.923604 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 16/32] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Aug 8 16:30:51 2022] POST /identity/v3/roles => generated 219 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:30:51.925210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743ddb3-431c-4703-975b-3f5579ae21f3 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:51.925600 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743ddb3-431c-4703-975b-3f5579ae21f3 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:51.925877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743ddb3-431c-4703-975b-3f5579ae21f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:51.952103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-b743ddb3-431c-4703-975b-3f5579ae21f3 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:30:51.953487 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-b743ddb3-431c-4703-975b-3f5579ae21f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:30:52.003502 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:52.018589 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36524d2e-8316-4b70-a475-50c268e9483d None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:52.046747 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 17/33] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:30:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:30:52.048305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743ddb3-431c-4703-975b-3f5579ae21f3 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:52.115445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b950d0-e819-4d7c-8d1b-1f92b62b1cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:52.116523 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b950d0-e819-4d7c-8d1b-1f92b62b1cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:52.117025 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b950d0-e819-4d7c-8d1b-1f92b62b1cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:52.117510 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b950d0-e819-4d7c-8d1b-1f92b62b1cc0 None admin] PATH_INFO: `/v3/users/b308731f39014d7098d27d547158b386` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:52.119557 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b950d0-e819-4d7c-8d1b-1f92b62b1cc0 None admin] RBAC: Authorizing `identity:get_user(user_id=b308731f39014d7098d27d547158b386)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:52.122013 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b950d0-e819-4d7c-8d1b-1f92b62b1cc0 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:52.124153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 17/34] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:30:51 2022] GET /identity/v3/users/b308731f39014d7098d27d547158b386 => generated 247 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:30:52.125386 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b950d0-e819-4d7c-8d1b-1f92b62b1cc0 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:52.198262 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743ddb3-431c-4703-975b-3f5579ae21f3 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:52.226772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:52.240446 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb8701-000d-4ce9-b935-c34208a68d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:52.241393 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb8701-000d-4ce9-b935-c34208a68d59 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:52.241814 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb8701-000d-4ce9-b935-c34208a68d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:52.242229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb8701-000d-4ce9-b935-c34208a68d59 None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:52.243084 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb8701-000d-4ce9-b935-c34208a68d59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:30:52.243587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb8701-000d-4ce9-b935-c34208a68d59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:52.245570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb8701-000d-4ce9-b935-c34208a68d59 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:52.266320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b950d0-e819-4d7c-8d1b-1f92b62b1cc0 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:52.266801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 18/35] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:30:52.267893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb8701-000d-4ce9-b935-c34208a68d59 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:52.315433 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010471a-1ef8-45e5-aa2a-585fb3e529f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:52.316558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010471a-1ef8-45e5-aa2a-585fb3e529f1 None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:52.317044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010471a-1ef8-45e5-aa2a-585fb3e529f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:52.317522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010471a-1ef8-45e5-aa2a-585fb3e529f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:52.318432 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e010471a-1ef8-45e5-aa2a-585fb3e529f1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:52.320278 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e010471a-1ef8-45e5-aa2a-585fb3e529f1 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:52.411018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb8701-000d-4ce9-b935-c34208a68d59 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:52.453437 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b37eaa0-b958-429a-a58e-47d23188ab99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:52.454508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37eaa0-b958-429a-a58e-47d23188ab99 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:52.454956 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37eaa0-b958-429a-a58e-47d23188ab99 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:52.455393 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37eaa0-b958-429a-a58e-47d23188ab99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:52.456794 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b37eaa0-b958-429a-a58e-47d23188ab99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:52.458470 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b37eaa0-b958-429a-a58e-47d23188ab99 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:52.460444 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 19/36] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:30:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:30:52.461383 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b37eaa0-b958-429a-a58e-47d23188ab99 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:52.540682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:52.558610 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 18/37] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Aug 8 16:30:52 2022] POST /identity/v3/roles => generated 225 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:30:52.558610 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010471a-1ef8-45e5-aa2a-585fb3e529f1 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:52.587346 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:52.609187 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b37eaa0-b958-429a-a58e-47d23188ab99 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:52.655937 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b07c98-fe08-4fd2-8d44-26f924a9748e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:52.657477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b07c98-fe08-4fd2-8d44-26f924a9748e None admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:52.657477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b07c98-fe08-4fd2-8d44-26f924a9748e None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:52.657477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b07c98-fe08-4fd2-8d44-26f924a9748e None admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:52.657835 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b07c98-fe08-4fd2-8d44-26f924a9748e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:52.659332 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b07c98-fe08-4fd2-8d44-26f924a9748e None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:52.691525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010471a-1ef8-45e5-aa2a-585fb3e529f1 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:52.730118 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b4472-a250-4f02-ba8c-37d0aaa49a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:52.730848 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b4472-a250-4f02-ba8c-37d0aaa49a7c None admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:52.731143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b4472-a250-4f02-ba8c-37d0aaa49a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:52.731425 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b4472-a250-4f02-ba8c-37d0aaa49a7c None admin] PATH_INFO: `/v3/domains/default/users/b308731f39014d7098d27d547158b386/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:52.733619 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b4472-a250-4f02-ba8c-37d0aaa49a7c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b308731f39014d7098d27d547158b386, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:52.735254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b4472-a250-4f02-ba8c-37d0aaa49a7c None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:52.764872 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 20/38] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Aug 8 16:30:52 2022] POST /identity/v3/projects => generated 301 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:30:52.766293 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054c122-9490-42c4-a302-8822c1f915d9 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:52.766603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054c122-9490-42c4-a302-8822c1f915d9 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:52.766887 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054c122-9490-42c4-a302-8822c1f915d9 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:52.768171 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 21/39] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:30:52.769340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e02876-10df-4ed4-9d11-ddce31be9b56 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:52.769689 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e02876-10df-4ed4-9d11-ddce31be9b56 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:52.769970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e02876-10df-4ed4-9d11-ddce31be9b56 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:52.771469 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 22/40] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:30:52.777840 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c109c-54c8-4e76-b68e-bc0bd3a513dd None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:52.778149 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c109c-54c8-4e76-b68e-bc0bd3a513dd None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:52.778393 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c109c-54c8-4e76-b68e-bc0bd3a513dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:52.799269 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-753c109c-54c8-4e76-b68e-bc0bd3a513dd None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:30:52.800399 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-753c109c-54c8-4e76-b68e-bc0bd3a513dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:30:52.816732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 19/41] 10.222.0.13 () {58 vars in 1361 bytes} [Mon Aug 8 16:30:52 2022] PUT /identity/v3/domains/default/users/b308731f39014d7098d27d547158b386/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:30:52.820340 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002e685-df73-4509-ac96-c7fbb8a8d9b1 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:52.820642 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002e685-df73-4509-ac96-c7fbb8a8d9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:52.820941 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002e685-df73-4509-ac96-c7fbb8a8d9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:52.854426 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-8002e685-df73-4509-ac96-c7fbb8a8d9b1 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:30:52.856617 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-8002e685-df73-4509-ac96-c7fbb8a8d9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:30:52.940832 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 23/42] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:30:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:30:52.968858 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c109c-54c8-4e76-b68e-bc0bd3a513dd None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:52.978309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 20/43] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:30:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:30:53.002015 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002e685-df73-4509-ac96-c7fbb8a8d9b1 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:53.013968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:53.022522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:53.100914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c109c-54c8-4e76-b68e-bc0bd3a513dd None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:53.123607 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002e685-df73-4509-ac96-c7fbb8a8d9b1 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:53.146346 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff43f6-565f-4f59-82fc-9c8033c6fa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:53.146985 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff43f6-565f-4f59-82fc-9c8033c6fa7d None admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:53.147264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff43f6-565f-4f59-82fc-9c8033c6fa7d None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:53.147535 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff43f6-565f-4f59-82fc-9c8033c6fa7d None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:53.148196 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff43f6-565f-4f59-82fc-9c8033c6fa7d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:53.149606 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff43f6-565f-4f59-82fc-9c8033c6fa7d None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:53.164063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a2aea-b992-4184-98d9-ec7df9eb548b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:53.165537 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a2aea-b992-4184-98d9-ec7df9eb548b None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:53.165537 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a2aea-b992-4184-98d9-ec7df9eb548b None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:53.165537 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a2aea-b992-4184-98d9-ec7df9eb548b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:53.166638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a2aea-b992-4184-98d9-ec7df9eb548b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:53.168746 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a2aea-b992-4184-98d9-ec7df9eb548b None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:53.170322 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 21/44] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:30:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:30:53.176003 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a2aea-b992-4184-98d9-ec7df9eb548b None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:53.238430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 24/45] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Aug 8 16:30:52 2022] POST /identity/v3/roles => generated 223 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:30:53.279021 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:53.317174 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a2aea-b992-4184-98d9-ec7df9eb548b None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:53.373358 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842a27f-c051-4489-ad7a-a88ef2da7025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:53.374009 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842a27f-c051-4489-ad7a-a88ef2da7025 None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:53.374280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842a27f-c051-4489-ad7a-a88ef2da7025 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:53.374557 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842a27f-c051-4489-ad7a-a88ef2da7025 None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:53.375273 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2842a27f-c051-4489-ad7a-a88ef2da7025 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:53.376810 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2842a27f-c051-4489-ad7a-a88ef2da7025 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:53.477422 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 22/46] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Aug 8 16:30:53 2022] POST /identity/v3/projects => generated 312 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:30:54.937663 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d0349-9444-457d-81bf-7f594b0f66ec None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:54.938116 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d0349-9444-457d-81bf-7f594b0f66ec None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:54.938228 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d0349-9444-457d-81bf-7f594b0f66ec None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:54.939661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 25/47] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:30:54.945037 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75bd62-3e74-4e3c-b771-06c06ac7038b None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:54.945443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75bd62-3e74-4e3c-b771-06c06ac7038b None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:54.945715 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75bd62-3e74-4e3c-b771-06c06ac7038b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:54.963458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-ce75bd62-3e74-4e3c-b771-06c06ac7038b None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:30:54.964669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ce75bd62-3e74-4e3c-b771-06c06ac7038b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:30:55.031230 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 23/48] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:30:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:30:55.055123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d0349-9444-457d-81bf-7f594b0f66ec None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:55.153925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:55.165263 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d0349-9444-457d-81bf-7f594b0f66ec None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:55.201698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce886a-a50c-4bc9-93eb-6baaf0b5d716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:55.201698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce886a-a50c-4bc9-93eb-6baaf0b5d716 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:55.201698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce886a-a50c-4bc9-93eb-6baaf0b5d716 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:55.202029 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce886a-a50c-4bc9-93eb-6baaf0b5d716 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:55.213406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce886a-a50c-4bc9-93eb-6baaf0b5d716 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:30:55.218720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-ebce886a-a50c-4bc9-93eb-6baaf0b5d716 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:30:55.218720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:30:55.218720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:30:55.218720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:30:55.218720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:30:55.218720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:30:55.218720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:30:55.218720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:30:55.218720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:30:55.218720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:30:55.218720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:30:55.218720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:30:55.218720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:30:55.218720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:30:55.218720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:30:55.218720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:30:55.220410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:30:55.220410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:30:55.220410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:30:55.220410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:30:55.220410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:30:55.220410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:30:55.220410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:30:55.220410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:30:55.220410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:30:55.220410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:30:55.220410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:30:55.220410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:30:55.220410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:30:55.220410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:30:55.220410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:30:55.220410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:30:55.221824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:30:55.221824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:30:55.221824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:30:55.221824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:30:55.221824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:30:55.221824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:30:55.221824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:30:55.221824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:30:55.221824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:30:55.221824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:30:55.221824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:30:55.221824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:30:55.221824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 26/49] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:30:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:30:55.226988 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75bd62-3e74-4e3c-b771-06c06ac7038b None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:55.288005 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:55.330766 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75bd62-3e74-4e3c-b771-06c06ac7038b None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:55.365214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8bb48-8ed3-4b83-8cc1-c0eb256941ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:55.365827 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8bb48-8ed3-4b83-8cc1-c0eb256941ed None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:55.366107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8bb48-8ed3-4b83-8cc1-c0eb256941ed None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:55.366378 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8bb48-8ed3-4b83-8cc1-c0eb256941ed None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:55.367078 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb8bb48-8ed3-4b83-8cc1-c0eb256941ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:30:55.367381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb8bb48-8ed3-4b83-8cc1-c0eb256941ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:55.368779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb8bb48-8ed3-4b83-8cc1-c0eb256941ed None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:55.377397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 24/50] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:30:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:30:55.382944 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce886a-a50c-4bc9-93eb-6baaf0b5d716 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:55.483270 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce886a-a50c-4bc9-93eb-6baaf0b5d716 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:55.524648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3f74b-bda4-426d-83c5-548ba20e1e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:55.525292 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3f74b-bda4-426d-83c5-548ba20e1e82 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:55.525568 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3f74b-bda4-426d-83c5-548ba20e1e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:55.525839 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3f74b-bda4-426d-83c5-548ba20e1e82 None admin] PATH_INFO: `/v3/users/b308731f39014d7098d27d547158b386` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:55.527019 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3f74b-bda4-426d-83c5-548ba20e1e82 None admin] RBAC: Authorizing `identity:get_user(user_id=b308731f39014d7098d27d547158b386)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:55.528501 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3f74b-bda4-426d-83c5-548ba20e1e82 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:55.530158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 27/51] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:30:55 2022] GET /identity/v3/users/b308731f39014d7098d27d547158b386 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:30:55.536427 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8bb48-8ed3-4b83-8cc1-c0eb256941ed None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:55.592767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52153d89-557c-46a0-b3b7-dbdd1d7d12ad None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:55.593300 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52153d89-557c-46a0-b3b7-dbdd1d7d12ad None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:55.593797 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52153d89-557c-46a0-b3b7-dbdd1d7d12ad None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:55.595209 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 28/52] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:30:55.601013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cf628-96a6-4e4e-932a-78b2c0c74466 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:55.601295 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cf628-96a6-4e4e-932a-78b2c0c74466 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:55.601563 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cf628-96a6-4e4e-932a-78b2c0c74466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:55.621810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-0a0cf628-96a6-4e4e-932a-78b2c0c74466 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:30:55.623089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-0a0cf628-96a6-4e4e-932a-78b2c0c74466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:30:55.640408 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:55.665438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8bb48-8ed3-4b83-8cc1-c0eb256941ed None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:55.685461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:55.690736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 29/53] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:30:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:30:55.699868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9f7b1-d5ff-4ff0-9a97-edacc8168e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:55.700504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9f7b1-d5ff-4ff0-9a97-edacc8168e1e None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:55.700783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9f7b1-d5ff-4ff0-9a97-edacc8168e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:55.701053 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9f7b1-d5ff-4ff0-9a97-edacc8168e1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:55.702157 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9f7b1-d5ff-4ff0-9a97-edacc8168e1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:55.703643 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9f7b1-d5ff-4ff0-9a97-edacc8168e1e None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:55.705207 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 25/54] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:30:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:30:55.710912 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cf628-96a6-4e4e-932a-78b2c0c74466 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:55.713554 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9f7b1-d5ff-4ff0-9a97-edacc8168e1e None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:55.813058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cf628-96a6-4e4e-932a-78b2c0c74466 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:55.816339 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9f7b1-d5ff-4ff0-9a97-edacc8168e1e None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:55.847212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf214a-a66f-49f5-aa04-9685553f3fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:55.847814 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf214a-a66f-49f5-aa04-9685553f3fde None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:55.848092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf214a-a66f-49f5-aa04-9685553f3fde None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:55.848366 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf214a-a66f-49f5-aa04-9685553f3fde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:55.849083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf214a-a66f-49f5-aa04-9685553f3fde None admin] RBAC: Adding query filter params (role.id=e578ef831127463d9b48cc89fe20329c, scope.domain.id=default, user.id=b308731f39014d7098d27d547158b386) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:30:55.849384 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf214a-a66f-49f5-aa04-9685553f3fde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:55.849897 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0248f4-f820-4e07-aa81-c49a23fb115d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:55.850534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0248f4-f820-4e07-aa81-c49a23fb115d None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:55.850855 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0248f4-f820-4e07-aa81-c49a23fb115d None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:55.850939 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf214a-a66f-49f5-aa04-9685553f3fde None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:55.851134 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0248f4-f820-4e07-aa81-c49a23fb115d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:55.852253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0248f4-f820-4e07-aa81-c49a23fb115d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:55.854325 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0248f4-f820-4e07-aa81-c49a23fb115d None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:55.856562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 26/55] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:30:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:30:55.857468 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 30/56] 10.222.0.13 () {56 vars in 1322 bytes} [Mon Aug 8 16:30:55 2022] GET /identity/v3/role_assignments?user.id=b308731f39014d7098d27d547158b386&scope.domain.id=default&role.id=e578ef831127463d9b48cc89fe20329c => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:30:55.862315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0248f4-f820-4e07-aa81-c49a23fb115d None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:55.925875 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:55.973417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0248f4-f820-4e07-aa81-c49a23fb115d None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:56.008444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de473be-e83e-4451-b1cd-576dcdb41277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:56.009064 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de473be-e83e-4451-b1cd-576dcdb41277 None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:56.009348 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de473be-e83e-4451-b1cd-576dcdb41277 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:56.009619 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de473be-e83e-4451-b1cd-576dcdb41277 None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:56.010273 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de473be-e83e-4451-b1cd-576dcdb41277 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:56.012047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de473be-e83e-4451-b1cd-576dcdb41277 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:56.113211 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 27/57] 10.222.0.13 () {60 vars in 1140 bytes} [Mon Aug 8 16:30:55 2022] POST /identity/v3/projects => generated 298 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:30:58.134636 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a291e7-2eed-43e6-b3d7-ea3b6baabbd1 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:58.135200 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a291e7-2eed-43e6-b3d7-ea3b6baabbd1 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:58.135200 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a291e7-2eed-43e6-b3d7-ea3b6baabbd1 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:58.136587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 31/58] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:30:58.141687 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d001315-a794-4066-a9c4-252061a506c9 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:58.142081 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d001315-a794-4066-a9c4-252061a506c9 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:58.142232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d001315-a794-4066-a9c4-252061a506c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:58.159972 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-4d001315-a794-4066-a9c4-252061a506c9 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:30:58.161163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-4d001315-a794-4066-a9c4-252061a506c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:30:58.227518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 28/59] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:30:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:30:58.252260 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a291e7-2eed-43e6-b3d7-ea3b6baabbd1 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:58.270394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:58.353092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a291e7-2eed-43e6-b3d7-ea3b6baabbd1 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:58.388316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5415e95-0970-49fd-8153-223b58aafe14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:58.388949 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5415e95-0970-49fd-8153-223b58aafe14 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:58.389228 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5415e95-0970-49fd-8153-223b58aafe14 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:58.389497 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5415e95-0970-49fd-8153-223b58aafe14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:58.390628 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5415e95-0970-49fd-8153-223b58aafe14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:58.392136 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5415e95-0970-49fd-8153-223b58aafe14 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:58.394243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 32/60] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:30:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:30:58.399996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d001315-a794-4066-a9c4-252061a506c9 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:30:58.439947 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:30:58.499022 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d001315-a794-4066-a9c4-252061a506c9 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:30:58.533070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c923158-c853-4a0f-9c53-7e6bcc734d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:30:58.533673 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c923158-c853-4a0f-9c53-7e6bcc734d89 None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:30:58.533954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c923158-c853-4a0f-9c53-7e6bcc734d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:30:58.534225 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c923158-c853-4a0f-9c53-7e6bcc734d89 None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:30:58.534909 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c923158-c853-4a0f-9c53-7e6bcc734d89 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:30:58.539589 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c923158-c853-4a0f-9c53-7e6bcc734d89 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:30:58.646140 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 29/61] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Aug 8 16:30:58 2022] POST /identity/v3/users => generated 275 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:31:01.781031 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d96f5-6c5a-419a-897e-9cc6c2a4d88c None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:01.781578 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d96f5-6c5a-419a-897e-9cc6c2a4d88c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:01.781578 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d96f5-6c5a-419a-897e-9cc6c2a4d88c None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:01.782946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 33/62] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:01.788335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53decfca-ee5e-4c6e-a18f-4869de9a12e3 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:01.788750 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53decfca-ee5e-4c6e-a18f-4869de9a12e3 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:01.788919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53decfca-ee5e-4c6e-a18f-4869de9a12e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:01.819799 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-53decfca-ee5e-4c6e-a18f-4869de9a12e3 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:01.821043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-53decfca-ee5e-4c6e-a18f-4869de9a12e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:01.976264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 30/63] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:02.016218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d96f5-6c5a-419a-897e-9cc6c2a4d88c None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:02.059948 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf52cf-abb6-45db-ada2-32122e636acf None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:02.060131 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf52cf-abb6-45db-ada2-32122e636acf None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:02.060467 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf52cf-abb6-45db-ada2-32122e636acf None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:02.061754 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 31/64] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:02.067694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243663b-0fc6-416b-9420-a794676bcb32 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:02.067982 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243663b-0fc6-416b-9420-a794676bcb32 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:02.068256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243663b-0fc6-416b-9420-a794676bcb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:02.102278 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-e243663b-0fc6-416b-9420-a794676bcb32 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:02.116964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-e243663b-0fc6-416b-9420-a794676bcb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:02.139931 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:02.170815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:02.195918 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d96f5-6c5a-419a-897e-9cc6c2a4d88c None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:02.207815 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 32/65] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:31:02.208116 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87541f2-07ea-461b-b18a-bb694f08f0e9 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:02.208331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87541f2-07ea-461b-b18a-bb694f08f0e9 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:02.208842 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87541f2-07ea-461b-b18a-bb694f08f0e9 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:02.210139 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 33/66] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:02.218334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03843f-eff5-4a1c-951b-435e0c7a216f None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:02.219980 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03843f-eff5-4a1c-951b-435e0c7a216f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:02.219980 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03843f-eff5-4a1c-951b-435e0c7a216f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:02.249130 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-ca03843f-eff5-4a1c-951b-435e0c7a216f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:02.251489 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ca03843f-eff5-4a1c-951b-435e0c7a216f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:02.259257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11448f-7131-47a6-8314-db13e27d1dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:02.260308 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11448f-7131-47a6-8314-db13e27d1dcc None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:02.261340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11448f-7131-47a6-8314-db13e27d1dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:02.261794 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11448f-7131-47a6-8314-db13e27d1dcc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:02.275767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11448f-7131-47a6-8314-db13e27d1dcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:02.284164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-6b11448f-7131-47a6-8314-db13e27d1dcc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:02.284164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:02.284164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:02.284164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:02.284164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:02.284164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:02.284164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:02.284164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:02.284164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:02.284164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:02.284164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:02.284164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:02.284164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:02.284164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:02.284164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:02.284164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:02.285109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:02.285109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:02.285109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:02.285109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:02.285109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:02.285109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:02.285109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:02.285109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:02.285109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:02.285109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:02.285109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:02.285109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:02.285109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:02.285109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:02.285109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:02.285109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:02.285899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:02.285899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:02.285899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:02.285899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:02.285899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:02.285899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:02.285899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:02.285899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:02.285899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:02.285899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:02.285899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:02.285899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:02.285899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 34/67] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:02.287439 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11448f-7131-47a6-8314-db13e27d1dcc None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:02.346648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 34/68] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:02.351521 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03843f-eff5-4a1c-951b-435e0c7a216f None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:02.417952 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11448f-7131-47a6-8314-db13e27d1dcc None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:02.455819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f30486-9f75-4d31-8869-7b5b6ae4c331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:02.456883 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f30486-9f75-4d31-8869-7b5b6ae4c331 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:02.457372 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f30486-9f75-4d31-8869-7b5b6ae4c331 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:02.457867 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:02.458354 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f30486-9f75-4d31-8869-7b5b6ae4c331 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:02.471432 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f30486-9f75-4d31-8869-7b5b6ae4c331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:02.476246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03843f-eff5-4a1c-951b-435e0c7a216f None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:02.476757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-d0f30486-9f75-4d31-8869-7b5b6ae4c331 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:31:02.476757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:02.476757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:02.476757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:02.476757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:02.476757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:02.476757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:02.476757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:02.476757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:02.476757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:02.476757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:02.476757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:02.476757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:02.476757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:02.476757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:02.476757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:02.477570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:02.477570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:02.477570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:02.477570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:02.477570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:02.477570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:02.477570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:02.477570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:02.477570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:02.477570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:02.477570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:02.477570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:02.477570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:02.477570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:02.477570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:02.477570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:02.478441 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:02.478441 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:02.478441 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:02.478441 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:02.478441 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:02.478441 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:02.478441 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:02.478441 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:02.478441 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:02.478441 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:02.478441 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:31:02.478441 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:02.481498 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 35/69] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:31:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:02.482552 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f30486-9f75-4d31-8869-7b5b6ae4c331 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:02.487346 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:02.519751 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3e93a-043b-49fb-817b-02cc13227321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:02.520373 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3e93a-043b-49fb-817b-02cc13227321 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:02.520655 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3e93a-043b-49fb-817b-02cc13227321 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:02.520931 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3e93a-043b-49fb-817b-02cc13227321 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:02.521947 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a3e93a-043b-49fb-817b-02cc13227321 None admin] RBAC: Adding query filter params (name=member) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:02.522271 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a3e93a-043b-49fb-817b-02cc13227321 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:02.524522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a3e93a-043b-49fb-817b-02cc13227321 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:02.535792 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 35/70] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:02.536682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3e93a-043b-49fb-817b-02cc13227321 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:02.603749 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f30486-9f75-4d31-8869-7b5b6ae4c331 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:02.639334 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a2ba2-a5f2-4b87-ab00-37df58103f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:02.639960 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a2ba2-a5f2-4b87-ab00-37df58103f7a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:02.640236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a2ba2-a5f2-4b87-ab00-37df58103f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:02.640505 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a2ba2-a5f2-4b87-ab00-37df58103f7a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:02.644464 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3e93a-043b-49fb-817b-02cc13227321 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:02.650501 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a2ba2-a5f2-4b87-ab00-37df58103f7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:02.656094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-ed1a2ba2-a5f2-4b87-ab00-37df58103f7a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:02.656094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:02.656094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:02.656094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:02.656094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:02.656094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:02.656094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:02.656094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:02.656094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:02.656094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:02.656094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:02.656094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:02.656094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:02.656094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:02.656094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:02.656094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:02.656977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:02.656977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:02.656977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:02.656977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:02.656977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:02.656977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:02.656977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:02.656977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:02.656977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:02.656977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:02.656977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:02.656977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:02.656977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:02.656977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:02.656977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:02.656977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:02.657780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:02.657780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:02.657780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:02.657780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:02.657780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:02.657780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:02.657780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:02.657780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:02.657780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:02.657780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:02.657780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:02.657780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:02.657780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 36/71] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Aug 8 16:31:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:02.659442 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a2ba2-a5f2-4b87-ab00-37df58103f7a None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:02.680177 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44aa037-d0fa-4e4e-a4ec-7127646a90cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:02.680803 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44aa037-d0fa-4e4e-a4ec-7127646a90cf None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:02.681085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44aa037-d0fa-4e4e-a4ec-7127646a90cf None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:02.681359 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44aa037-d0fa-4e4e-a4ec-7127646a90cf None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:02.682025 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44aa037-d0fa-4e4e-a4ec-7127646a90cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:02.682328 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44aa037-d0fa-4e4e-a4ec-7127646a90cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:02.684645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44aa037-d0fa-4e4e-a4ec-7127646a90cf None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:02.693264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 36/72] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:02.698034 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e65da5-82f3-47d5-bd15-ec0ca698baf6 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:02.698034 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e65da5-82f3-47d5-bd15-ec0ca698baf6 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:02.698034 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e65da5-82f3-47d5-bd15-ec0ca698baf6 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:02.698034 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 37/73] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:02.698034 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1db007-9e04-4f12-b593-7251b0aa521c None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:02.698034 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1db007-9e04-4f12-b593-7251b0aa521c None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:02.699308 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1db007-9e04-4f12-b593-7251b0aa521c None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:02.699915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 38/74] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:02.700368 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1db007-9e04-4f12-b593-7251b0aa521c None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:02.712004 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:02.730646 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:02.772639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a2ba2-a5f2-4b87-ab00-37df58103f7a None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:02.804893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1db007-9e04-4f12-b593-7251b0aa521c None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:02.807028 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d0b45-0007-4dd9-9e81-222f9047475f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:02.808384 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d0b45-0007-4dd9-9e81-222f9047475f None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:02.808836 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d0b45-0007-4dd9-9e81-222f9047475f None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:02.809046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d0b45-0007-4dd9-9e81-222f9047475f None admin] PATH_INFO: `/v3/users/db4e226a05124c368e8ad70f2a45b7ad` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:02.824229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d0b45-0007-4dd9-9e81-222f9047475f None admin] RBAC: Authorizing `identity:get_user(user_id=db4e226a05124c368e8ad70f2a45b7ad)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:02.825807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d0b45-0007-4dd9-9e81-222f9047475f None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:02.827503 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 37/75] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:02 2022] GET /identity/v3/users/db4e226a05124c368e8ad70f2a45b7ad => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:02.828274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d0b45-0007-4dd9-9e81-222f9047475f None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:02.841330 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9748b-9c40-4611-ab54-2b5f9d560491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:02.841948 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9748b-9c40-4611-ab54-2b5f9d560491 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:02.842222 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9748b-9c40-4611-ab54-2b5f9d560491 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:02.842489 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9748b-9c40-4611-ab54-2b5f9d560491 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:02.843484 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9748b-9c40-4611-ab54-2b5f9d560491 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:02.843784 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9748b-9c40-4611-ab54-2b5f9d560491 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:02.845142 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9748b-9c40-4611-ab54-2b5f9d560491 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:02.852923 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 39/76] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:31:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:02.854105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba80e7-94c7-4eb7-9afa-afb6be7be79d None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:02.854386 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba80e7-94c7-4eb7-9afa-afb6be7be79d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:02.854675 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba80e7-94c7-4eb7-9afa-afb6be7be79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:02.872844 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-64ba80e7-94c7-4eb7-9afa-afb6be7be79d None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:02.874093 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-64ba80e7-94c7-4eb7-9afa-afb6be7be79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:02.932047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d0b45-0007-4dd9-9e81-222f9047475f None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:02.937265 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:02.941747 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 40/77] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:02.942992 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22195ff3-24de-406f-a52e-6fefbde64939 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:02.943278 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22195ff3-24de-406f-a52e-6fefbde64939 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:02.943548 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22195ff3-24de-406f-a52e-6fefbde64939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:02.961921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-22195ff3-24de-406f-a52e-6fefbde64939 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:02.963490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-22195ff3-24de-406f-a52e-6fefbde64939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:02.966514 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:02.978284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755bbe3-296c-47d2-a0da-2c3cdf18f8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:02.979044 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755bbe3-296c-47d2-a0da-2c3cdf18f8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:02.983064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755bbe3-296c-47d2-a0da-2c3cdf18f8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:02.983064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755bbe3-296c-47d2-a0da-2c3cdf18f8b4 None admin] PATH_INFO: `/v3/users/b308731f39014d7098d27d547158b386` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:02.983064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755bbe3-296c-47d2-a0da-2c3cdf18f8b4 None admin] RBAC: Authorizing `identity:get_user(user_id=b308731f39014d7098d27d547158b386)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:02.983064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755bbe3-296c-47d2-a0da-2c3cdf18f8b4 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:02.983927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 38/78] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:02 2022] GET /identity/v3/users/b308731f39014d7098d27d547158b386 => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:02.985114 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755bbe3-296c-47d2-a0da-2c3cdf18f8b4 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:03.031885 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 41/79] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:03.032810 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22195ff3-24de-406f-a52e-6fefbde64939 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:03.087863 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755bbe3-296c-47d2-a0da-2c3cdf18f8b4 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:03.122680 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147bca7a-265c-4ed9-85d2-e2d8735df23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:03.123293 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147bca7a-265c-4ed9-85d2-e2d8735df23a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:03.123573 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147bca7a-265c-4ed9-85d2-e2d8735df23a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:03.123866 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147bca7a-265c-4ed9-85d2-e2d8735df23a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:03.140232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22195ff3-24de-406f-a52e-6fefbde64939 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:03.170180 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:03.176432 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e84af-5c50-4cd2-a449-7fe8f7307a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:03.177099 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e84af-5c50-4cd2-a449-7fe8f7307a06 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:03.177381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e84af-5c50-4cd2-a449-7fe8f7307a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:03.177655 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e84af-5c50-4cd2-a449-7fe8f7307a06 None admin] PATH_INFO: `/v3/users/db4e226a05124c368e8ad70f2a45b7ad` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:03.178840 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e84af-5c50-4cd2-a449-7fe8f7307a06 None admin] RBAC: Authorizing `identity:get_user(user_id=db4e226a05124c368e8ad70f2a45b7ad)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:03.180312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e84af-5c50-4cd2-a449-7fe8f7307a06 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:03.182179 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 42/80] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:03 2022] GET /identity/v3/users/db4e226a05124c368e8ad70f2a45b7ad => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:03.183439 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e84af-5c50-4cd2-a449-7fe8f7307a06 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:03.184687 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:03.195162 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147bca7a-265c-4ed9-85d2-e2d8735df23a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:03.196642 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147bca7a-265c-4ed9-85d2-e2d8735df23a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:03.287924 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e84af-5c50-4cd2-a449-7fe8f7307a06 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:03.297057 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 39/81] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:03.297903 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147bca7a-265c-4ed9-85d2-e2d8735df23a None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:03.322589 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce8ff4-195c-412b-b6ab-8571cc07d2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:03.323491 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce8ff4-195c-412b-b6ab-8571cc07d2fb None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:03.323771 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce8ff4-195c-412b-b6ab-8571cc07d2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:03.324123 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce8ff4-195c-412b-b6ab-8571cc07d2fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:03.335276 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce8ff4-195c-412b-b6ab-8571cc07d2fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:03.341808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-6bce8ff4-195c-412b-b6ab-8571cc07d2fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:03.341808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:03.341808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:03.341808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:03.341808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:03.341808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:03.341808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:03.341808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:03.341808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:03.341808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:03.341808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:03.341808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:03.341808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:03.341808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:03.341808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:03.341808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:03.343182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:03.343182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:03.343182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:03.343182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:03.343182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:03.343182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:03.343182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:03.343182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:03.343182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:03.343182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:03.343182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:03.343182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:03.343182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:03.343182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:03.343182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:03.343182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:03.343951 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:03.343951 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:03.343951 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:03.343951 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:03.343951 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:03.343951 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:03.343951 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:03.343951 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:03.343951 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:03.343951 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:03.343951 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:03.343951 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:03.343951 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 43/82] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:03.344685 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce8ff4-195c-412b-b6ab-8571cc07d2fb None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:03.390498 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:03.399697 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147bca7a-265c-4ed9-85d2-e2d8735df23a None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:03.406546 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:03.433878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ee59d-529c-4f23-ac10-34db260b6ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:03.434483 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ee59d-529c-4f23-ac10-34db260b6ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:03.434782 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ee59d-529c-4f23-ac10-34db260b6ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:03.435049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ee59d-529c-4f23-ac10-34db260b6ce6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:03.444783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce8ff4-195c-412b-b6ab-8571cc07d2fb None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:03.478972 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc2d65-d50a-4c92-a845-5adfdfb5c8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:03.480231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc2d65-d50a-4c92-a845-5adfdfb5c8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:03.480582 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc2d65-d50a-4c92-a845-5adfdfb5c8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:03.480890 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc2d65-d50a-4c92-a845-5adfdfb5c8f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:03.482149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dc2d65-d50a-4c92-a845-5adfdfb5c8f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:03.484088 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dc2d65-d50a-4c92-a845-5adfdfb5c8f5 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:03.485430 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 44/83] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:31:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:03.486204 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc2d65-d50a-4c92-a845-5adfdfb5c8f5 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:03.503615 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ee59d-529c-4f23-ac10-34db260b6ce6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:03.505103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ee59d-529c-4f23-ac10-34db260b6ce6 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:03.586879 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc2d65-d50a-4c92-a845-5adfdfb5c8f5 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:03.594027 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:03.604099 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 40/84] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:03.605745 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ee59d-529c-4f23-ac10-34db260b6ce6 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:03.613986 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:03.623148 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa832c-2e86-4230-8b01-251bc017e20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:03.623759 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa832c-2e86-4230-8b01-251bc017e20c None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:03.624037 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa832c-2e86-4230-8b01-251bc017e20c None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:03.624302 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa832c-2e86-4230-8b01-251bc017e20c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:03.692805 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa832c-2e86-4230-8b01-251bc017e20c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:03.694211 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa832c-2e86-4230-8b01-251bc017e20c None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:03.704057 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ee59d-529c-4f23-ac10-34db260b6ce6 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:03.741435 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590e498-8e2b-42e7-b751-d6291bf9fa5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:03.742070 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590e498-8e2b-42e7-b751-d6291bf9fa5b None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:03.742341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590e498-8e2b-42e7-b751-d6291bf9fa5b None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:03.742681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590e498-8e2b-42e7-b751-d6291bf9fa5b None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:03.743736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b590e498-8e2b-42e7-b751-d6291bf9fa5b None admin] RBAC: Authorizing `identity:get_project(project_id=88933471b063476fb90fe841b4e9b75b)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:03.745236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b590e498-8e2b-42e7-b751-d6291bf9fa5b None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:03.746886 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 41/85] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:03 2022] GET /identity/v3/projects/88933471b063476fb90fe841b4e9b75b => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:03.747644 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590e498-8e2b-42e7-b751-d6291bf9fa5b None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:03.794924 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 45/86] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:03.795758 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa832c-2e86-4230-8b01-251bc017e20c None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:03.805965 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:03.821691 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:03.849560 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590e498-8e2b-42e7-b751-d6291bf9fa5b None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:03.883908 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d1581-eb9c-4756-9a55-6d4ecfa8037c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:03.884511 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d1581-eb9c-4756-9a55-6d4ecfa8037c None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:03.884848 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d1581-eb9c-4756-9a55-6d4ecfa8037c None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:03.885126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d1581-eb9c-4756-9a55-6d4ecfa8037c None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:03.886799 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d1581-eb9c-4756-9a55-6d4ecfa8037c None admin] RBAC: Adding query filter params (name=member) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:03.887133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d1581-eb9c-4756-9a55-6d4ecfa8037c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:03.888520 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d1581-eb9c-4756-9a55-6d4ecfa8037c None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:03.893614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa832c-2e86-4230-8b01-251bc017e20c None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:03.896628 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 42/87] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:03.897426 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d1581-eb9c-4756-9a55-6d4ecfa8037c None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:03.932171 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad7069-e536-43f4-8e5a-17ae13a509d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:03.933346 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad7069-e536-43f4-8e5a-17ae13a509d4 None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:03.933484 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad7069-e536-43f4-8e5a-17ae13a509d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:03.933771 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad7069-e536-43f4-8e5a-17ae13a509d4 None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:03.934521 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad7069-e536-43f4-8e5a-17ae13a509d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:03.937033 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad7069-e536-43f4-8e5a-17ae13a509d4 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:04.004602 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d1581-eb9c-4756-9a55-6d4ecfa8037c None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:04.024671 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:04.038892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9459948-ed94-4a09-becf-96ec0cbcaafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:04.039552 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9459948-ed94-4a09-becf-96ec0cbcaafb None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:04.039830 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9459948-ed94-4a09-becf-96ec0cbcaafb None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:04.040100 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9459948-ed94-4a09-becf-96ec0cbcaafb None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:04.041194 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9459948-ed94-4a09-becf-96ec0cbcaafb None admin] RBAC: Authorizing `identity:get_project(project_id=88933471b063476fb90fe841b4e9b75b)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:04.043255 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9459948-ed94-4a09-becf-96ec0cbcaafb None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:04.044876 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 43/88] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:03 2022] GET /identity/v3/projects/88933471b063476fb90fe841b4e9b75b => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:04.045639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9459948-ed94-4a09-becf-96ec0cbcaafb None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:04.055486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 46/89] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Aug 8 16:31:03 2022] POST /identity/v3/users => generated 289 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:31:04.055486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad7069-e536-43f4-8e5a-17ae13a509d4 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:04.150934 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:04.170815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9459948-ed94-4a09-becf-96ec0cbcaafb None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:04.171853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad7069-e536-43f4-8e5a-17ae13a509d4 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:04.212161 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3642abb-8409-4197-9f99-0d713fa48a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:04.212776 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3642abb-8409-4197-9f99-0d713fa48a1e None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:04.213051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3642abb-8409-4197-9f99-0d713fa48a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:04.213319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3642abb-8409-4197-9f99-0d713fa48a1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:04.214265 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3642abb-8409-4197-9f99-0d713fa48a1e None admin] RBAC: Adding query filter params (role.id=04bf40fcc26347de8167fc215f62c7aa, scope.project.id=88933471b063476fb90fe841b4e9b75b, user.id=db4e226a05124c368e8ad70f2a45b7ad) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:04.214707 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3642abb-8409-4197-9f99-0d713fa48a1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:04.216049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3642abb-8409-4197-9f99-0d713fa48a1e None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:04.216657 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2571f3-d3d2-4e25-aaf5-421bb49850f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:04.217309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2571f3-d3d2-4e25-aaf5-421bb49850f1 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:04.217593 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2571f3-d3d2-4e25-aaf5-421bb49850f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:04.217866 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2571f3-d3d2-4e25-aaf5-421bb49850f1 None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:04.219296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2571f3-d3d2-4e25-aaf5-421bb49850f1 None admin] RBAC: Authorizing `identity:get_project(project_id=88933471b063476fb90fe841b4e9b75b)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:04.221645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2571f3-d3d2-4e25-aaf5-421bb49850f1 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:04.222390 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 44/90] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:31:04 2022] GET /identity/v3/role_assignments?user.id=db4e226a05124c368e8ad70f2a45b7ad&scope.project.id=88933471b063476fb90fe841b4e9b75b&role.id=04bf40fcc26347de8167fc215f62c7aa => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:04.223243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3642abb-8409-4197-9f99-0d713fa48a1e None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:04.227418 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 47/91] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:04 2022] GET /identity/v3/projects/88933471b063476fb90fe841b4e9b75b => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:04.227687 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2571f3-d3d2-4e25-aaf5-421bb49850f1 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:04.295818 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:04.375329 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2571f3-d3d2-4e25-aaf5-421bb49850f1 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:04.377264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3642abb-8409-4197-9f99-0d713fa48a1e None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:04.424751 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc460e-9df0-4718-9660-b9f54eb339ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:04.425421 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc460e-9df0-4718-9660-b9f54eb339ba None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:04.425720 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc460e-9df0-4718-9660-b9f54eb339ba None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:04.425968 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20ea07-3993-4676-b150-4bb6e27e0eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:04.426087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc460e-9df0-4718-9660-b9f54eb339ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:04.426632 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20ea07-3993-4676-b150-4bb6e27e0eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:04.426743 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc460e-9df0-4718-9660-b9f54eb339ba None admin] RBAC: Adding query filter params (role.id=e578ef831127463d9b48cc89fe20329c, scope.project.id=88933471b063476fb90fe841b4e9b75b, user.id=b308731f39014d7098d27d547158b386) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:04.426909 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20ea07-3993-4676-b150-4bb6e27e0eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:04.427055 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc460e-9df0-4718-9660-b9f54eb339ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:04.427198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20ea07-3993-4676-b150-4bb6e27e0eb3 None admin] PATH_INFO: `/v3/users/db4e226a05124c368e8ad70f2a45b7ad` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:04.428587 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc460e-9df0-4718-9660-b9f54eb339ba None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:04.429093 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e20ea07-3993-4676-b150-4bb6e27e0eb3 None admin] RBAC: Authorizing `identity:get_user(user_id=db4e226a05124c368e8ad70f2a45b7ad)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:04.430541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e20ea07-3993-4676-b150-4bb6e27e0eb3 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:04.432201 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 45/92] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:04 2022] GET /identity/v3/users/db4e226a05124c368e8ad70f2a45b7ad => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:04.434622 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20ea07-3993-4676-b150-4bb6e27e0eb3 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:04.439905 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 48/93] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:31:04 2022] GET /identity/v3/role_assignments?user.id=b308731f39014d7098d27d547158b386&scope.project.id=88933471b063476fb90fe841b4e9b75b&role.id=e578ef831127463d9b48cc89fe20329c => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:04.440927 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc460e-9df0-4718-9660-b9f54eb339ba None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:04.460382 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:04.556191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20ea07-3993-4676-b150-4bb6e27e0eb3 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:04.592489 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:04.598454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d227890-bce6-4313-ab4d-ca05014c48cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:04.599256 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d227890-bce6-4313-ab4d-ca05014c48cf None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:04.599635 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d227890-bce6-4313-ab4d-ca05014c48cf None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:04.600004 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d227890-bce6-4313-ab4d-ca05014c48cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:04.600958 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d227890-bce6-4313-ab4d-ca05014c48cf None admin] RBAC: Adding query filter params (role.id=fd2599a3f408404599ecea94959efc4a, scope.project.id=88933471b063476fb90fe841b4e9b75b, user.id=db4e226a05124c368e8ad70f2a45b7ad) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:04.601361 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d227890-bce6-4313-ab4d-ca05014c48cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:04.605304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d227890-bce6-4313-ab4d-ca05014c48cf None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:04.610223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 46/94] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:31:04 2022] GET /identity/v3/role_assignments?user.id=db4e226a05124c368e8ad70f2a45b7ad&scope.project.id=88933471b063476fb90fe841b4e9b75b&role.id=fd2599a3f408404599ecea94959efc4a => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:04.641557 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc460e-9df0-4718-9660-b9f54eb339ba None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:04.681154 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26154aa5-7576-40bb-a20e-93004b5a0a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:04.681790 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26154aa5-7576-40bb-a20e-93004b5a0a18 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:04.682074 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26154aa5-7576-40bb-a20e-93004b5a0a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:04.682414 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26154aa5-7576-40bb-a20e-93004b5a0a18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:04.762473 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26154aa5-7576-40bb-a20e-93004b5a0a18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:04.764042 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26154aa5-7576-40bb-a20e-93004b5a0a18 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:04.791114 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:04.874967 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 49/95] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:04.880988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d227890-bce6-4313-ab4d-ca05014c48cf None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:05.027671 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d227890-bce6-4313-ab4d-ca05014c48cf None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:05.076579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881fa6c-9f45-4776-af6b-fa2664c00aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:05.077222 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881fa6c-9f45-4776-af6b-fa2664c00aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:05.077503 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881fa6c-9f45-4776-af6b-fa2664c00aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:05.077777 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881fa6c-9f45-4776-af6b-fa2664c00aa2 None admin] PATH_INFO: `/v3/projects/9fecbed0db964b66a7ddfa4d0f4df31c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:05.078950 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881fa6c-9f45-4776-af6b-fa2664c00aa2 None admin] RBAC: Authorizing `identity:get_project(project_id=9fecbed0db964b66a7ddfa4d0f4df31c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:05.080399 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881fa6c-9f45-4776-af6b-fa2664c00aa2 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:05.082032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 47/96] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:04 2022] GET /identity/v3/projects/9fecbed0db964b66a7ddfa4d0f4df31c => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:05.088181 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26154aa5-7576-40bb-a20e-93004b5a0a18 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:05.221235 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26154aa5-7576-40bb-a20e-93004b5a0a18 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:05.241344 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:05.270288 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532adfd-91c8-4599-a421-7308ece4f609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:05.270940 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532adfd-91c8-4599-a421-7308ece4f609 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:05.271223 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532adfd-91c8-4599-a421-7308ece4f609 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:05.271497 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532adfd-91c8-4599-a421-7308ece4f609 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:05.272234 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532adfd-91c8-4599-a421-7308ece4f609 None admin] RBAC: Adding query filter params (role.id=04bf40fcc26347de8167fc215f62c7aa, scope.project.id=9fecbed0db964b66a7ddfa4d0f4df31c, user.id=db4e226a05124c368e8ad70f2a45b7ad) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:05.272539 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532adfd-91c8-4599-a421-7308ece4f609 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:05.273993 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532adfd-91c8-4599-a421-7308ece4f609 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:05.280616 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 50/97] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:31:05 2022] GET /identity/v3/role_assignments?user.id=db4e226a05124c368e8ad70f2a45b7ad&scope.project.id=9fecbed0db964b66a7ddfa4d0f4df31c&role.id=04bf40fcc26347de8167fc215f62c7aa => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:06.838227 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a8172-7763-452d-ae12-19d3590e0482 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:06.838831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a8172-7763-452d-ae12-19d3590e0482 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:06.838831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a8172-7763-452d-ae12-19d3590e0482 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:06.840480 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 48/98] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:06.846237 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa68b9-d661-4033-b5d6-04892c03f3fb None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:06.846714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa68b9-d661-4033-b5d6-04892c03f3fb None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:06.846901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa68b9-d661-4033-b5d6-04892c03f3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:06.871618 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-67aa68b9-d661-4033-b5d6-04892c03f3fb None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:06.872843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-67aa68b9-d661-4033-b5d6-04892c03f3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:06.966248 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 51/99] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:07.005503 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a8172-7763-452d-ae12-19d3590e0482 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:07.078158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:07.144215 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a8172-7763-452d-ae12-19d3590e0482 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:07.186671 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869aecb-6d70-495f-a529-e29f9d22c1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:07.187650 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869aecb-6d70-495f-a529-e29f9d22c1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:07.188160 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869aecb-6d70-495f-a529-e29f9d22c1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:07.188650 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869aecb-6d70-495f-a529-e29f9d22c1b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:07.189959 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869aecb-6d70-495f-a529-e29f9d22c1b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:07.191704 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869aecb-6d70-495f-a529-e29f9d22c1b8 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:07.193476 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 49/100] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:31:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:07.200717 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa68b9-d661-4033-b5d6-04892c03f3fb None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:07.212650 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464331d-f156-47b1-bd24-1f762d90ca83 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:07.214429 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464331d-f156-47b1-bd24-1f762d90ca83 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:07.216510 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464331d-f156-47b1-bd24-1f762d90ca83 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:07.218316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 50/101] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:07.224392 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e728a1d-0a80-4f03-a5fb-2d497f7008e5 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:07.224943 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e728a1d-0a80-4f03-a5fb-2d497f7008e5 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:07.225412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e728a1d-0a80-4f03-a5fb-2d497f7008e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:07.262288 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-8e728a1d-0a80-4f03-a5fb-2d497f7008e5 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:07.263913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-8e728a1d-0a80-4f03-a5fb-2d497f7008e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:07.379442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa68b9-d661-4033-b5d6-04892c03f3fb None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:07.419285 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:07.436125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 51/102] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:07.438170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226b30b-bbb7-4445-8d78-a07c75872e50 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:07.438725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226b30b-bbb7-4445-8d78-a07c75872e50 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:07.439241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226b30b-bbb7-4445-8d78-a07c75872e50 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:07.441273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 52/103] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:07.462513 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04a6cb-9a9c-4c4d-be4e-553a36db684d None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:07.463121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04a6cb-9a9c-4c4d-be4e-553a36db684d None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:07.463662 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04a6cb-9a9c-4c4d-be4e-553a36db684d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:07.477789 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1d97b-1edc-4dcf-ad30-062c759f9e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:07.478426 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1d97b-1edc-4dcf-ad30-062c759f9e9f None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:07.478731 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1d97b-1edc-4dcf-ad30-062c759f9e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:07.479001 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1d97b-1edc-4dcf-ad30-062c759f9e9f None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:07.479701 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a1d97b-1edc-4dcf-ad30-062c759f9e9f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:07.481288 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a1d97b-1edc-4dcf-ad30-062c759f9e9f None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:07.555546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-4f04a6cb-9a9c-4c4d-be4e-553a36db684d None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:07.557116 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-4f04a6cb-9a9c-4c4d-be4e-553a36db684d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:07.584852 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 52/104] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Aug 8 16:31:07 2022] POST /identity/v3/projects => generated 302 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:31:07.587669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1d97b-1edc-4dcf-ad30-062c759f9e9f None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:07.639572 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:07.670398 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 53/105] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:07.672118 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6518b96-010d-4a84-8912-c9da7de187c2 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:07.672625 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6518b96-010d-4a84-8912-c9da7de187c2 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:07.673101 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6518b96-010d-4a84-8912-c9da7de187c2 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:07.674651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 54/106] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:07.688116 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbade65-a2a5-431e-95e9-f5dc655cfdf4 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:07.688547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbade65-a2a5-431e-95e9-f5dc655cfdf4 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:07.688832 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbade65-a2a5-431e-95e9-f5dc655cfdf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:07.708227 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-5dbade65-a2a5-431e-95e9-f5dc655cfdf4 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:07.709435 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-5dbade65-a2a5-431e-95e9-f5dc655cfdf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:07.798148 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 55/107] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:07.798853 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbade65-a2a5-431e-95e9-f5dc655cfdf4 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:07.826906 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1d97b-1edc-4dcf-ad30-062c759f9e9f None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:07.868964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74daf37b-bad9-4193-91e3-28b473faf337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:07.869739 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74daf37b-bad9-4193-91e3-28b473faf337 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:07.870490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74daf37b-bad9-4193-91e3-28b473faf337 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:07.870490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74daf37b-bad9-4193-91e3-28b473faf337 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:07.882725 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74daf37b-bad9-4193-91e3-28b473faf337 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:07.888561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-74daf37b-bad9-4193-91e3-28b473faf337 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:31:07.888561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:07.888561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:07.888561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:07.888561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:07.888561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:07.888561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:07.888561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:07.888561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:07.888561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:07.888561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:07.888561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:07.888561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:07.888561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:07.888561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:07.888561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:07.889443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:07.889443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:07.889443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:07.889443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:07.889443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:07.889443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:07.889443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:07.889443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:07.889443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:07.889443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:07.889443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:07.889443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:07.889443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:07.889443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:07.889443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:07.889443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:07.890198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:07.890198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:07.890198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:07.890198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:07.890198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:07.890198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:07.890198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:07.890198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:07.890198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:07.890198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:07.890198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:31:07.890198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:07.890198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 53/108] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:31:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:07.890956 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cedc2-d161-419a-8a45-f961cc1d028b None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:07.892087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cedc2-d161-419a-8a45-f961cc1d028b None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:07.892087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cedc2-d161-419a-8a45-f961cc1d028b None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:07.892895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 54/109] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:07.894188 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857cedc2-d161-419a-8a45-f961cc1d028b None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:07.918177 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:07.940858 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbade65-a2a5-431e-95e9-f5dc655cfdf4 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:07.946976 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:07.972544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf962e11-5a9d-4c59-86b9-549528b4b84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:07.973306 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf962e11-5a9d-4c59-86b9-549528b4b84e None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:07.973585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf962e11-5a9d-4c59-86b9-549528b4b84e None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:07.973904 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf962e11-5a9d-4c59-86b9-549528b4b84e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:07.988597 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf962e11-5a9d-4c59-86b9-549528b4b84e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:07.995916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-bf962e11-5a9d-4c59-86b9-549528b4b84e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:07.995916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:07.995916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:07.995916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:07.995916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:07.995916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:07.995916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:07.995916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:07.995916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:07.995916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:07.995916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:07.995916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:07.995916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:07.995916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:07.995916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:07.995916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:07.996846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:07.996846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:07.996846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:07.996846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:07.996846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:07.996846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:07.996846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:07.996846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:07.996846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:07.996846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:07.996846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:07.996846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:07.996846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:07.996846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:07.996846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:07.996846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:07.997617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:07.997617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:07.997617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:07.997617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:07.997617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:07.997617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:07.997617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:07.997617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:07.997617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:07.997617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:07.997617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:07.997617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:07.997617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 56/110] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:07.998265 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf962e11-5a9d-4c59-86b9-549528b4b84e None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:08.022312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857cedc2-d161-419a-8a45-f961cc1d028b None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:08.070914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bd00d-83c3-4a99-a374-c207befc19f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:08.072207 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bd00d-83c3-4a99-a374-c207befc19f8 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:08.073754 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bd00d-83c3-4a99-a374-c207befc19f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:08.074452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bd00d-83c3-4a99-a374-c207befc19f8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:08.089060 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76bd00d-83c3-4a99-a374-c207befc19f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:08.096353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-e76bd00d-83c3-4a99-a374-c207befc19f8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:31:08.096353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:08.096353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:08.096353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:08.096353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:08.096353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:08.096353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:08.096353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:08.096353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:08.096353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:08.096353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:08.096353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:08.096353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:08.096353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:08.096353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:08.096353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:08.097264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:08.097264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:08.097264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:08.097264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:08.097264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:08.097264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:08.097264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:08.097264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:08.097264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:08.097264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:08.097264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:08.097264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:08.097264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:08.097264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:08.097264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:08.097264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:08.098020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:08.098020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:08.098020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:08.098020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:08.098020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:08.098020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:08.098020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:08.098020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:08.098020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:08.098020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:08.098020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:31:08.098020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:08.099996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 55/111] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:08.105775 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06608b-1305-47d1-8d4a-b9a450d628a0 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:08.105775 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06608b-1305-47d1-8d4a-b9a450d628a0 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:08.105775 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06608b-1305-47d1-8d4a-b9a450d628a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:08.128775 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-ea06608b-1305-47d1-8d4a-b9a450d628a0 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:08.129320 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf962e11-5a9d-4c59-86b9-549528b4b84e None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:08.130002 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ea06608b-1305-47d1-8d4a-b9a450d628a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:08.179741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e66599-0106-4dbe-bfab-bfe2cd66f991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:08.181173 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e66599-0106-4dbe-bfab-bfe2cd66f991 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:08.181648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e66599-0106-4dbe-bfab-bfe2cd66f991 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:08.182689 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e66599-0106-4dbe-bfab-bfe2cd66f991 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:08.183872 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e66599-0106-4dbe-bfab-bfe2cd66f991 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:08.184411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e66599-0106-4dbe-bfab-bfe2cd66f991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:08.186738 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e66599-0106-4dbe-bfab-bfe2cd66f991 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:08.198596 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 57/112] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:08.200335 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e66599-0106-4dbe-bfab-bfe2cd66f991 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:08.214591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 56/113] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:08.214591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea06608b-1305-47d1-8d4a-b9a450d628a0 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:08.230553 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:08.252172 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:08.331418 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e66599-0106-4dbe-bfab-bfe2cd66f991 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:08.363945 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea06608b-1305-47d1-8d4a-b9a450d628a0 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:08.370328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c94dc-bae4-45b0-af04-69aaa2cecdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:08.371149 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c94dc-bae4-45b0-af04-69aaa2cecdb0 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:08.371531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c94dc-bae4-45b0-af04-69aaa2cecdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:08.371906 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c94dc-bae4-45b0-af04-69aaa2cecdb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:08.372682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c94dc-bae4-45b0-af04-69aaa2cecdb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:08.373099 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c94dc-bae4-45b0-af04-69aaa2cecdb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:08.374592 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c94dc-bae4-45b0-af04-69aaa2cecdb0 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:08.383905 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 58/114] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:08.384705 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c94dc-bae4-45b0-af04-69aaa2cecdb0 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:08.413340 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90a3c2-2a57-448a-9e9d-8081d8755bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:08.414504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90a3c2-2a57-448a-9e9d-8081d8755bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:08.415053 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90a3c2-2a57-448a-9e9d-8081d8755bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:08.415708 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90a3c2-2a57-448a-9e9d-8081d8755bc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:08.416974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a90a3c2-2a57-448a-9e9d-8081d8755bc5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:08.417421 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a90a3c2-2a57-448a-9e9d-8081d8755bc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:08.419594 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a90a3c2-2a57-448a-9e9d-8081d8755bc5 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:08.436242 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 57/115] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:08.436242 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90a3c2-2a57-448a-9e9d-8081d8755bc5 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:08.499702 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:08.507549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c94dc-bae4-45b0-af04-69aaa2cecdb0 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:08.536334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:08.545232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d0500-ed78-4e1f-b07a-30dbfea2830e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:08.546255 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d0500-ed78-4e1f-b07a-30dbfea2830e None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:08.546934 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d0500-ed78-4e1f-b07a-30dbfea2830e None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:08.547704 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d0500-ed78-4e1f-b07a-30dbfea2830e None admin] PATH_INFO: `/v3/users/b308731f39014d7098d27d547158b386` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:08.548973 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d0500-ed78-4e1f-b07a-30dbfea2830e None admin] RBAC: Authorizing `identity:get_user(user_id=b308731f39014d7098d27d547158b386)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:08.550642 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d0500-ed78-4e1f-b07a-30dbfea2830e None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:08.552571 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 59/116] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:08 2022] GET /identity/v3/users/b308731f39014d7098d27d547158b386 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:08.553373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d0500-ed78-4e1f-b07a-30dbfea2830e None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:08.576626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90a3c2-2a57-448a-9e9d-8081d8755bc5 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:08.613345 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48eaf9-34d7-43a4-aae8-223497f0dc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:08.613979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48eaf9-34d7-43a4-aae8-223497f0dc0d None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:08.614254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48eaf9-34d7-43a4-aae8-223497f0dc0d None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:08.614522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48eaf9-34d7-43a4-aae8-223497f0dc0d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:08.626442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48eaf9-34d7-43a4-aae8-223497f0dc0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:08.631782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-2e48eaf9-34d7-43a4-aae8-223497f0dc0d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:08.631782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:08.631782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:08.631782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:08.631782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:08.631782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:08.631782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:08.631782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:08.631782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:08.631782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:08.631782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:08.631782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:08.631782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:08.631782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:08.631782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:08.631782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:08.632779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:08.632779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:08.632779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:08.632779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:08.632779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:08.632779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:08.632779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:08.632779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:08.632779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:08.632779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:08.632779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:08.632779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:08.632779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:08.632779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:08.632779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:08.632779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:08.633627 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:08.633627 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:08.633627 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:08.633627 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:08.633627 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:08.633627 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:08.633627 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:08.633627 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:08.633627 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:08.633627 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:08.633627 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:08.633627 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:08.633627 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 58/117] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Aug 8 16:31:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:08.634282 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48eaf9-34d7-43a4-aae8-223497f0dc0d None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:08.670415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d0500-ed78-4e1f-b07a-30dbfea2830e None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:08.707784 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab75890-0ee6-4278-81da-1c0db829c661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:08.708416 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab75890-0ee6-4278-81da-1c0db829c661 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:08.708696 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab75890-0ee6-4278-81da-1c0db829c661 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:08.708967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab75890-0ee6-4278-81da-1c0db829c661 None admin] PATH_INFO: `/v3/users/db4e226a05124c368e8ad70f2a45b7ad` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:08.710105 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab75890-0ee6-4278-81da-1c0db829c661 None admin] RBAC: Authorizing `identity:get_user(user_id=db4e226a05124c368e8ad70f2a45b7ad)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:08.711604 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab75890-0ee6-4278-81da-1c0db829c661 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:08.714026 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 60/118] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:08 2022] GET /identity/v3/users/db4e226a05124c368e8ad70f2a45b7ad => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:08.716059 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab75890-0ee6-4278-81da-1c0db829c661 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:08.743312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48eaf9-34d7-43a4-aae8-223497f0dc0d None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:08.751958 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:08.780031 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:08.786776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc94de-3bb0-4bc9-a772-2eefb285b66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:08.787440 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc94de-3bb0-4bc9-a772-2eefb285b66f None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:08.787704 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc94de-3bb0-4bc9-a772-2eefb285b66f None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:08.787975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc94de-3bb0-4bc9-a772-2eefb285b66f None admin] PATH_INFO: `/v3/users/1b6faeb56200440caf14f97686b3425b` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:08.808797 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cc94de-3bb0-4bc9-a772-2eefb285b66f None admin] RBAC: Authorizing `identity:get_user(user_id=1b6faeb56200440caf14f97686b3425b)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:08.810473 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cc94de-3bb0-4bc9-a772-2eefb285b66f None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:08.816557 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 59/119] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:08 2022] GET /identity/v3/users/1b6faeb56200440caf14f97686b3425b => generated 289 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:08.816557 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a86d9-770f-4814-b664-e2cf8edcfb5d None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:08.816557 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a86d9-770f-4814-b664-e2cf8edcfb5d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:08.816557 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a86d9-770f-4814-b664-e2cf8edcfb5d None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:08.816557 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 60/120] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:08.816557 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a86d9-770f-4814-b664-e2cf8edcfb5d None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:08.833004 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab75890-0ee6-4278-81da-1c0db829c661 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:08.879571 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8980ec-ff98-46dd-a48d-805793df1e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:08.880234 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8980ec-ff98-46dd-a48d-805793df1e59 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:08.880517 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8980ec-ff98-46dd-a48d-805793df1e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:08.880854 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8980ec-ff98-46dd-a48d-805793df1e59 None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:08.882049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8980ec-ff98-46dd-a48d-805793df1e59 None admin] RBAC: Authorizing `identity:get_project(project_id=88933471b063476fb90fe841b4e9b75b)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:08.883557 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8980ec-ff98-46dd-a48d-805793df1e59 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:08.885290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 61/121] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:08 2022] GET /identity/v3/projects/88933471b063476fb90fe841b4e9b75b => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:08.886269 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8980ec-ff98-46dd-a48d-805793df1e59 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:08.940419 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a86d9-770f-4814-b664-e2cf8edcfb5d None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:08.976582 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921ba03-9e3c-42ec-8c31-0b4d2f4ce66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:08.977503 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921ba03-9e3c-42ec-8c31-0b4d2f4ce66f None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:08.977786 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921ba03-9e3c-42ec-8c31-0b4d2f4ce66f None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:08.978055 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921ba03-9e3c-42ec-8c31-0b4d2f4ce66f None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:08.978768 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921ba03-9e3c-42ec-8c31-0b4d2f4ce66f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:08.979239 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921ba03-9e3c-42ec-8c31-0b4d2f4ce66f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:08.981228 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921ba03-9e3c-42ec-8c31-0b4d2f4ce66f None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:08.989503 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 61/122] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:31:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:08.990313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921ba03-9e3c-42ec-8c31-0b4d2f4ce66f None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:09.000339 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:09.004167 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8980ec-ff98-46dd-a48d-805793df1e59 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:09.040148 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b453c1b-4f73-4a78-acd5-f6d3c4bb93f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:09.040776 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b453c1b-4f73-4a78-acd5-f6d3c4bb93f7 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:09.041054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b453c1b-4f73-4a78-acd5-f6d3c4bb93f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:09.041325 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b453c1b-4f73-4a78-acd5-f6d3c4bb93f7 None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:09.043051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b453c1b-4f73-4a78-acd5-f6d3c4bb93f7 None admin] RBAC: Authorizing `identity:get_project(project_id=88933471b063476fb90fe841b4e9b75b)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:09.044507 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b453c1b-4f73-4a78-acd5-f6d3c4bb93f7 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:09.046487 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 62/123] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:08 2022] GET /identity/v3/projects/88933471b063476fb90fe841b4e9b75b => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:09.047742 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3e660-affa-499f-ab2b-24f36ec95d2c None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:09.048374 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3e660-affa-499f-ab2b-24f36ec95d2c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:09.049134 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3e660-affa-499f-ab2b-24f36ec95d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:09.062590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:09.070364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-1cd3e660-affa-499f-ab2b-24f36ec95d2c None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:09.071584 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-1cd3e660-affa-499f-ab2b-24f36ec95d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:09.108437 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921ba03-9e3c-42ec-8c31-0b4d2f4ce66f None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:09.139379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 63/124] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:09.140196 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3e660-affa-499f-ab2b-24f36ec95d2c None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:09.154558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051979b-7f93-45a7-81b6-e9cda75bcdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:09.156088 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051979b-7f93-45a7-81b6-e9cda75bcdbb None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:09.156670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051979b-7f93-45a7-81b6-e9cda75bcdbb None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:09.157142 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051979b-7f93-45a7-81b6-e9cda75bcdbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:09.233537 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051979b-7f93-45a7-81b6-e9cda75bcdbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:09.235057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051979b-7f93-45a7-81b6-e9cda75bcdbb None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:09.240064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:09.246791 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3e660-affa-499f-ab2b-24f36ec95d2c None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:09.285610 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c53d2-7ff5-42f3-a4d1-0919346dc085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:09.286187 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c53d2-7ff5-42f3-a4d1-0919346dc085 None admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:09.286467 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c53d2-7ff5-42f3-a4d1-0919346dc085 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:09.286776 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c53d2-7ff5-42f3-a4d1-0919346dc085 None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b/users/b308731f39014d7098d27d547158b386/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:09.288756 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c53d2-7ff5-42f3-a4d1-0919346dc085 None admin] RBAC: Authorizing `identity:create_grant(project_id=88933471b063476fb90fe841b4e9b75b, user_id=b308731f39014d7098d27d547158b386, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:09.290205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c53d2-7ff5-42f3-a4d1-0919346dc085 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:09.300569 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:09.347139 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 62/125] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:09.347987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051979b-7f93-45a7-81b6-e9cda75bcdbb None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:09.368447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 64/126] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:31:09 2022] PUT /identity/v3/projects/88933471b063476fb90fe841b4e9b75b/users/b308731f39014d7098d27d547158b386/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:31:09.369646 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c53d2-7ff5-42f3-a4d1-0919346dc085 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:09.485448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051979b-7f93-45a7-81b6-e9cda75bcdbb None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:09.525486 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c53d2-7ff5-42f3-a4d1-0919346dc085 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:09.526543 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199cb27d-5512-4e26-be32-1d2111201ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:09.527575 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cb27d-5512-4e26-be32-1d2111201ace None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:09.528060 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cb27d-5512-4e26-be32-1d2111201ace None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:09.528522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cb27d-5512-4e26-be32-1d2111201ace None admin] PATH_INFO: `/v3/users/db4e226a05124c368e8ad70f2a45b7ad` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:09.530054 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199cb27d-5512-4e26-be32-1d2111201ace None admin] RBAC: Authorizing `identity:get_user(user_id=db4e226a05124c368e8ad70f2a45b7ad)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:09.531866 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199cb27d-5512-4e26-be32-1d2111201ace None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:09.538129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 63/127] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:09 2022] GET /identity/v3/users/db4e226a05124c368e8ad70f2a45b7ad => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:09.539549 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199cb27d-5512-4e26-be32-1d2111201ace None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:09.567005 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:09.567217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a5c47-c4e2-4df7-bb00-e3b35b843991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:09.567741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a5c47-c4e2-4df7-bb00-e3b35b843991 None admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:09.568150 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a5c47-c4e2-4df7-bb00-e3b35b843991 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:09.568340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a5c47-c4e2-4df7-bb00-e3b35b843991 None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b/users/db4e226a05124c368e8ad70f2a45b7ad/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:09.570240 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a5c47-c4e2-4df7-bb00-e3b35b843991 None admin] RBAC: Authorizing `identity:create_grant(project_id=88933471b063476fb90fe841b4e9b75b, user_id=db4e226a05124c368e8ad70f2a45b7ad, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:09.571751 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a5c47-c4e2-4df7-bb00-e3b35b843991 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:09.652505 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199cb27d-5512-4e26-be32-1d2111201ace None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:09.694594 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 65/128] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:31:09 2022] PUT /identity/v3/projects/88933471b063476fb90fe841b4e9b75b/users/db4e226a05124c368e8ad70f2a45b7ad/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:31:09.694594 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a5c47-c4e2-4df7-bb00-e3b35b843991 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:09.713563 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48cade1-c9ef-470a-8e0f-306133c08ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:09.714477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cade1-c9ef-470a-8e0f-306133c08ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:09.714786 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cade1-c9ef-470a-8e0f-306133c08ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:09.715057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cade1-c9ef-470a-8e0f-306133c08ac0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:09.733486 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:09.740066 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48cade1-c9ef-470a-8e0f-306133c08ac0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:09.749820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-a48cade1-c9ef-470a-8e0f-306133c08ac0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:09.749820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:09.749820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:09.749820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:09.749820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:09.749820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:09.749820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:09.749820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:09.749820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:09.749820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:09.749820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:09.749820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:09.749820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:09.749820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:09.749820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:09.749820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:09.750816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:09.750816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:09.750816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:09.750816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:09.750816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:09.750816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:09.750816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:09.750816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:09.750816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:09.750816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:09.750816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:09.750816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:09.750816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:09.750816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:09.750816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:09.750816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:09.751633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:09.751633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:09.751633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:09.751633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:09.751633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:09.751633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:09.751633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:09.751633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:09.751633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:09.751633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:09.751633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:09.751633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:09.752226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 64/129] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:09.753164 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48cade1-c9ef-470a-8e0f-306133c08ac0 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:09.883245 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a5c47-c4e2-4df7-bb00-e3b35b843991 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:09.893264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:09.927946 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48cade1-c9ef-470a-8e0f-306133c08ac0 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:09.929759 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0d875-e007-4d2d-82aa-6f182abef27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:09.930632 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0d875-e007-4d2d-82aa-6f182abef27b None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:09.931121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0d875-e007-4d2d-82aa-6f182abef27b None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:09.932042 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0d875-e007-4d2d-82aa-6f182abef27b None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:09.934380 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0d875-e007-4d2d-82aa-6f182abef27b None admin] RBAC: Authorizing `identity:get_project(project_id=88933471b063476fb90fe841b4e9b75b)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:09.935196 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0d875-e007-4d2d-82aa-6f182abef27b None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:09.937134 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 66/130] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:09 2022] GET /identity/v3/projects/88933471b063476fb90fe841b4e9b75b => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:09.943222 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0d875-e007-4d2d-82aa-6f182abef27b None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:09.973147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383bb2aa-7ba1-4c68-ab6c-a266e5a76ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:09.974516 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bb2aa-7ba1-4c68-ab6c-a266e5a76ddf None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:09.975102 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bb2aa-7ba1-4c68-ab6c-a266e5a76ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:09.975572 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bb2aa-7ba1-4c68-ab6c-a266e5a76ddf None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:09.976954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383bb2aa-7ba1-4c68-ab6c-a266e5a76ddf None admin] RBAC: Authorizing `identity:get_project(project_id=88933471b063476fb90fe841b4e9b75b)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:09.978807 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383bb2aa-7ba1-4c68-ab6c-a266e5a76ddf None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:09.981413 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 65/131] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:09 2022] GET /identity/v3/projects/88933471b063476fb90fe841b4e9b75b => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:09.982983 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45ee13-1c4a-473e-9ce7-390dacca6dab None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:09.983513 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45ee13-1c4a-473e-9ce7-390dacca6dab None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:09.983957 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45ee13-1c4a-473e-9ce7-390dacca6dab None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:09.985508 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 66/132] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:09.986745 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45ee13-1c4a-473e-9ce7-390dacca6dab None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:10.039246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:10.056167 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0d875-e007-4d2d-82aa-6f182abef27b None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:10.093621 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2a305-ba39-4642-a7f3-e0803d033bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:10.094234 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2a305-ba39-4642-a7f3-e0803d033bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:10.094514 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2a305-ba39-4642-a7f3-e0803d033bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:10.094817 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2a305-ba39-4642-a7f3-e0803d033bf7 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:10.095500 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca2a305-ba39-4642-a7f3-e0803d033bf7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:10.095951 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca2a305-ba39-4642-a7f3-e0803d033bf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:10.097277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca2a305-ba39-4642-a7f3-e0803d033bf7 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:10.103353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45ee13-1c4a-473e-9ce7-390dacca6dab None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:10.111176 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 67/133] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:10.111176 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2a305-ba39-4642-a7f3-e0803d033bf7 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:10.148674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64fb2e8-1413-4595-b922-f81075904af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:10.149524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fb2e8-1413-4595-b922-f81075904af7 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:10.150089 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fb2e8-1413-4595-b922-f81075904af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:10.150504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fb2e8-1413-4595-b922-f81075904af7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:10.151556 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64fb2e8-1413-4595-b922-f81075904af7 None admin] RBAC: Adding query filter params (role.id=94f7f84c31e0425495dcca9783e7a46e, scope.project.id=88933471b063476fb90fe841b4e9b75b, user.id=1b6faeb56200440caf14f97686b3425b) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:10.152167 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64fb2e8-1413-4595-b922-f81075904af7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:10.153954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64fb2e8-1413-4595-b922-f81075904af7 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:10.160589 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 67/134] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:31:09 2022] GET /identity/v3/role_assignments?user.id=1b6faeb56200440caf14f97686b3425b&scope.project.id=88933471b063476fb90fe841b4e9b75b&role.id=94f7f84c31e0425495dcca9783e7a46e => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:10.162185 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75525ac-6ea3-4a23-90c2-e68507a9256c None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:10.162737 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75525ac-6ea3-4a23-90c2-e68507a9256c None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:10.163264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75525ac-6ea3-4a23-90c2-e68507a9256c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:10.186994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-b75525ac-6ea3-4a23-90c2-e68507a9256c None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:10.189019 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b75525ac-6ea3-4a23-90c2-e68507a9256c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:10.199608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:10.219528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2a305-ba39-4642-a7f3-e0803d033bf7 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:10.257973 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81791d-5873-4024-8f31-fc6ada5934e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:10.258814 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81791d-5873-4024-8f31-fc6ada5934e4 None admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:10.259130 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81791d-5873-4024-8f31-fc6ada5934e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:10.259408 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81791d-5873-4024-8f31-fc6ada5934e4 None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b/users/db4e226a05124c368e8ad70f2a45b7ad/roles/fd2599a3f408404599ecea94959efc4a` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:10.262017 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81791d-5873-4024-8f31-fc6ada5934e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=88933471b063476fb90fe841b4e9b75b, user_id=db4e226a05124c368e8ad70f2a45b7ad, role_id=fd2599a3f408404599ecea94959efc4a)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:10.263616 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81791d-5873-4024-8f31-fc6ada5934e4 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:10.271256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 68/135] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:10.272226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75525ac-6ea3-4a23-90c2-e68507a9256c None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:10.337890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 68/136] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:31:10 2022] PUT /identity/v3/projects/88933471b063476fb90fe841b4e9b75b/users/db4e226a05124c368e8ad70f2a45b7ad/roles/fd2599a3f408404599ecea94959efc4a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:31:10.347088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81791d-5873-4024-8f31-fc6ada5934e4 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:10.365665 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:10.391934 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75525ac-6ea3-4a23-90c2-e68507a9256c None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:10.431141 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc7f9d-7725-4e3d-a2a7-a7fb5560e14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:10.431709 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc7f9d-7725-4e3d-a2a7-a7fb5560e14c None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:10.431985 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc7f9d-7725-4e3d-a2a7-a7fb5560e14c None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:10.432279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc7f9d-7725-4e3d-a2a7-a7fb5560e14c None admin] PATH_INFO: `/v3/users/db4e226a05124c368e8ad70f2a45b7ad` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:10.433462 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc7f9d-7725-4e3d-a2a7-a7fb5560e14c None admin] RBAC: Authorizing `identity:get_user(user_id=db4e226a05124c368e8ad70f2a45b7ad)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:10.434980 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc7f9d-7725-4e3d-a2a7-a7fb5560e14c None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:10.439251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 69/137] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:10 2022] GET /identity/v3/users/db4e226a05124c368e8ad70f2a45b7ad => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:10.444253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc7f9d-7725-4e3d-a2a7-a7fb5560e14c None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:10.458976 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:10.557676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc7f9d-7725-4e3d-a2a7-a7fb5560e14c None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:10.576235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81791d-5873-4024-8f31-fc6ada5934e4 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:10.604751 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3af504-7aaf-4fc4-88ba-c09d5f092396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:10.605512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3af504-7aaf-4fc4-88ba-c09d5f092396 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:10.605802 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3af504-7aaf-4fc4-88ba-c09d5f092396 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:10.606078 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3af504-7aaf-4fc4-88ba-c09d5f092396 None admin] PATH_INFO: `/v3/projects/9fecbed0db964b66a7ddfa4d0f4df31c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:10.611427 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3af504-7aaf-4fc4-88ba-c09d5f092396 None admin] RBAC: Authorizing `identity:get_project(project_id=9fecbed0db964b66a7ddfa4d0f4df31c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:10.613056 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3af504-7aaf-4fc4-88ba-c09d5f092396 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:10.615002 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 70/138] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:10 2022] GET /identity/v3/projects/9fecbed0db964b66a7ddfa4d0f4df31c => generated 312 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:10.620852 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3af504-7aaf-4fc4-88ba-c09d5f092396 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:10.659292 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d8370-44f4-46d5-9a5b-7bf3ba21644e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:10.659940 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d8370-44f4-46d5-9a5b-7bf3ba21644e None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:10.660355 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d8370-44f4-46d5-9a5b-7bf3ba21644e None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:10.660542 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d8370-44f4-46d5-9a5b-7bf3ba21644e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:10.661710 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68d8370-44f4-46d5-9a5b-7bf3ba21644e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:10.663265 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68d8370-44f4-46d5-9a5b-7bf3ba21644e None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:10.665271 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 69/139] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:31:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:10.670656 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d8370-44f4-46d5-9a5b-7bf3ba21644e None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:10.726408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:10.749202 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3af504-7aaf-4fc4-88ba-c09d5f092396 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:10.787574 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb53119-f59e-44d5-9fb8-e0e161c08b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:10.788245 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb53119-f59e-44d5-9fb8-e0e161c08b8c None admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:10.788524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb53119-f59e-44d5-9fb8-e0e161c08b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:10.788796 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb53119-f59e-44d5-9fb8-e0e161c08b8c None admin] PATH_INFO: `/v3/projects/9fecbed0db964b66a7ddfa4d0f4df31c/users/db4e226a05124c368e8ad70f2a45b7ad/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:10.790927 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb53119-f59e-44d5-9fb8-e0e161c08b8c None admin] RBAC: Authorizing `identity:create_grant(project_id=9fecbed0db964b66a7ddfa4d0f4df31c, user_id=db4e226a05124c368e8ad70f2a45b7ad, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:10.792433 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb53119-f59e-44d5-9fb8-e0e161c08b8c None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:10.827112 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:10.855340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d8370-44f4-46d5-9a5b-7bf3ba21644e None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:10.880738 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 71/140] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:31:10 2022] PUT /identity/v3/projects/9fecbed0db964b66a7ddfa4d0f4df31c/users/db4e226a05124c368e8ad70f2a45b7ad/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:31:10.905715 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a41b14-b1af-430b-acb8-028a60c8b964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:10.906330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a41b14-b1af-430b-acb8-028a60c8b964 None admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:10.906626 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a41b14-b1af-430b-acb8-028a60c8b964 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:10.906898 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a41b14-b1af-430b-acb8-028a60c8b964 None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:10.907560 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a41b14-b1af-430b-acb8-028a60c8b964 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:10.909037 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a41b14-b1af-430b-acb8-028a60c8b964 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:11.012796 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 70/141] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Aug 8 16:31:10 2022] POST /identity/v3/users => generated 283 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:31:12.092483 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbfcc3-9cb6-4152-9621-12ffc18a4458 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:12.093269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbfcc3-9cb6-4152-9621-12ffc18a4458 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:12.093269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbfcc3-9cb6-4152-9621-12ffc18a4458 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:12.094641 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 72/142] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:12.099899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c185344-bc5a-4781-a9e9-4729c9dae43e None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:12.100364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c185344-bc5a-4781-a9e9-4729c9dae43e None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:12.100446 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c185344-bc5a-4781-a9e9-4729c9dae43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:12.124851 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-1c185344-bc5a-4781-a9e9-4729c9dae43e None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:12.126043 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-1c185344-bc5a-4781-a9e9-4729c9dae43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:12.201097 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 71/143] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:12.226736 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbfcc3-9cb6-4152-9621-12ffc18a4458 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:12.349556 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbfcc3-9cb6-4152-9621-12ffc18a4458 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:12.391509 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4688fd-a48c-4624-860a-832127c88598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:12.392160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4688fd-a48c-4624-860a-832127c88598 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:12.392436 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4688fd-a48c-4624-860a-832127c88598 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:12.392705 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4688fd-a48c-4624-860a-832127c88598 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:12.400075 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:12.404699 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4688fd-a48c-4624-860a-832127c88598 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:12.409629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-fd4688fd-a48c-4624-860a-832127c88598 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:12.409629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:12.409629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:12.409629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:12.409629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:12.409629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:12.409629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:12.409629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:12.409629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:12.409629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:12.409629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:12.409629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:12.409629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:12.409629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:12.409629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:12.409629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:12.410512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:12.410512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:12.410512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:12.410512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:12.410512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:12.410512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:12.410512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:12.410512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:12.410512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:12.410512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:12.410512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:12.410512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:12.410512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:12.410512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:12.410512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:12.410512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:12.411367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:12.411367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:12.411367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:12.411367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:12.411367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:12.411367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:12.411367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:12.411367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:12.411367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:12.411367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:12.411367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:12.411367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:12.413554 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 73/144] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:12.419020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c185344-bc5a-4781-a9e9-4729c9dae43e None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:12.494443 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:12.556069 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c185344-bc5a-4781-a9e9-4729c9dae43e None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:12.592709 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a6838-24df-468d-8950-d6b3a8c56f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:12.593635 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a6838-24df-468d-8950-d6b3a8c56f81 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:12.593917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a6838-24df-468d-8950-d6b3a8c56f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:12.594185 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a6838-24df-468d-8950-d6b3a8c56f81 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:12.594897 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a6838-24df-468d-8950-d6b3a8c56f81 None admin] RBAC: Adding query filter params (name=member) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:12.595196 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a6838-24df-468d-8950-d6b3a8c56f81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:12.596607 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a6838-24df-468d-8950-d6b3a8c56f81 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:12.611807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 72/145] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:12.616941 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4688fd-a48c-4624-860a-832127c88598 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:12.745277 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4688fd-a48c-4624-860a-832127c88598 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:12.788717 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd68a78-92d3-4a00-97e5-29546de72b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:12.789350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd68a78-92d3-4a00-97e5-29546de72b98 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:12.789622 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd68a78-92d3-4a00-97e5-29546de72b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:12.789923 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd68a78-92d3-4a00-97e5-29546de72b98 None admin] PATH_INFO: `/v3/users/db4e226a05124c368e8ad70f2a45b7ad` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:12.792253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd68a78-92d3-4a00-97e5-29546de72b98 None admin] RBAC: Authorizing `identity:get_user(user_id=db4e226a05124c368e8ad70f2a45b7ad)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:12.793764 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd68a78-92d3-4a00-97e5-29546de72b98 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:12.795476 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 74/146] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:12 2022] GET /identity/v3/users/db4e226a05124c368e8ad70f2a45b7ad => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:12.800998 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a6838-24df-468d-8950-d6b3a8c56f81 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:12.913701 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542c44d-8690-4a23-bdbf-8011c1a96052 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:12.914046 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542c44d-8690-4a23-bdbf-8011c1a96052 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:12.914236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542c44d-8690-4a23-bdbf-8011c1a96052 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:12.916689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 75/147] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:12.927361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5b079-2946-4dbe-89e9-411e8ea9941e None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:12.927920 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5b079-2946-4dbe-89e9-411e8ea9941e None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:12.928195 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5b079-2946-4dbe-89e9-411e8ea9941e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:12.933454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a6838-24df-468d-8950-d6b3a8c56f81 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:12.970084 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-d2e5b079-2946-4dbe-89e9-411e8ea9941e None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:12.971305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-d2e5b079-2946-4dbe-89e9-411e8ea9941e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:12.982412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:13.012860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1511b5-07b8-4851-b881-cdbc007c0050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:13.012860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1511b5-07b8-4851-b881-cdbc007c0050 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:13.012860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1511b5-07b8-4851-b881-cdbc007c0050 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:13.012860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1511b5-07b8-4851-b881-cdbc007c0050 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:13.103301 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1511b5-07b8-4851-b881-cdbc007c0050 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:13.105400 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1511b5-07b8-4851-b881-cdbc007c0050 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:13.132655 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:13.149071 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 76/148] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:31:13.239850 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 73/149] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:13.245863 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5b079-2946-4dbe-89e9-411e8ea9941e None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:13.248307 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1511b5-07b8-4851-b881-cdbc007c0050 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:13.275140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:13.393398 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1511b5-07b8-4851-b881-cdbc007c0050 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:13.400499 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5b079-2946-4dbe-89e9-411e8ea9941e None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:13.444691 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6e9b8-1ed2-4256-8461-9edda2f03b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:13.445859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6e9b8-1ed2-4256-8461-9edda2f03b98 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:13.446350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6e9b8-1ed2-4256-8461-9edda2f03b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:13.446847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6e9b8-1ed2-4256-8461-9edda2f03b98 None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:13.449697 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6e9b8-1ed2-4256-8461-9edda2f03b98 None admin] RBAC: Authorizing `identity:get_project(project_id=88933471b063476fb90fe841b4e9b75b)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:13.451531 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6e9b8-1ed2-4256-8461-9edda2f03b98 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:13.453524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 77/150] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:13 2022] GET /identity/v3/projects/88933471b063476fb90fe841b4e9b75b => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:13.459706 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6e9b8-1ed2-4256-8461-9edda2f03b98 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:13.483288 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929d901-ecfd-4ce6-b16b-17ad0a28a321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:13.484201 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929d901-ecfd-4ce6-b16b-17ad0a28a321 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:13.484630 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929d901-ecfd-4ce6-b16b-17ad0a28a321 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:13.485056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929d901-ecfd-4ce6-b16b-17ad0a28a321 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:13.515617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1929d901-ecfd-4ce6-b16b-17ad0a28a321 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:13.520654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-1929d901-ecfd-4ce6-b16b-17ad0a28a321 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:31:13.520654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:13.520654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:13.520654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:13.520654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:13.520654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:13.520654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:13.520654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:13.520654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:13.520654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:13.520654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:13.520654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:13.520654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:13.520654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:13.520654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:13.520654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:13.521674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:13.521674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:13.521674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:13.521674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:13.521674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:13.521674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:13.521674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:13.521674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:13.521674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:13.521674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:13.521674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:13.521674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:13.521674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:13.521674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:13.521674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:13.521674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:13.522454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:13.522454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:13.522454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:13.522454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:13.522454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:13.522454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:13.522454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:13.522454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:13.522454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:13.522454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:13.522454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:31:13.522454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:13.530662 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 74/151] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:31:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:13.531818 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929d901-ecfd-4ce6-b16b-17ad0a28a321 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:13.552731 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:13.599689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6e9b8-1ed2-4256-8461-9edda2f03b98 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:13.650381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a33edb7-b084-4d24-8a86-b65c73347492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:13.651342 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33edb7-b084-4d24-8a86-b65c73347492 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:13.651786 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33edb7-b084-4d24-8a86-b65c73347492 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:13.652302 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33edb7-b084-4d24-8a86-b65c73347492 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:13.653184 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a33edb7-b084-4d24-8a86-b65c73347492 None admin] RBAC: Adding query filter params (role.id=04bf40fcc26347de8167fc215f62c7aa, scope.project.id=88933471b063476fb90fe841b4e9b75b, user.id=db4e226a05124c368e8ad70f2a45b7ad) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:13.653731 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a33edb7-b084-4d24-8a86-b65c73347492 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:13.660047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a33edb7-b084-4d24-8a86-b65c73347492 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:13.667080 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 78/152] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:31:13 2022] GET /identity/v3/role_assignments?user.id=db4e226a05124c368e8ad70f2a45b7ad&scope.project.id=88933471b063476fb90fe841b4e9b75b&role.id=04bf40fcc26347de8167fc215f62c7aa => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:13.668761 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492e0a7-61d5-4b73-8dab-945699706bec None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:13.669276 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492e0a7-61d5-4b73-8dab-945699706bec None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:13.669692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492e0a7-61d5-4b73-8dab-945699706bec None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:13.671115 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 79/153] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:13.676881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e63ff2-53e2-4465-95ab-9058db2a1bda None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:13.677411 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e63ff2-53e2-4465-95ab-9058db2a1bda None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:13.677956 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e63ff2-53e2-4465-95ab-9058db2a1bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:13.711696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-a8e63ff2-53e2-4465-95ab-9058db2a1bda None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:13.713253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-a8e63ff2-53e2-4465-95ab-9058db2a1bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:13.758597 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:13.812964 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929d901-ecfd-4ce6-b16b-17ad0a28a321 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:13.815457 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 80/154] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:31:13.864365 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e63ff2-53e2-4465-95ab-9058db2a1bda None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:13.893440 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df9c0c-40e1-4c49-8845-32ce9c96cf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:13.894464 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df9c0c-40e1-4c49-8845-32ce9c96cf28 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:13.895005 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df9c0c-40e1-4c49-8845-32ce9c96cf28 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:13.895414 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df9c0c-40e1-4c49-8845-32ce9c96cf28 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:13.896241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df9c0c-40e1-4c49-8845-32ce9c96cf28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:13.896682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df9c0c-40e1-4c49-8845-32ce9c96cf28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:13.903692 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df9c0c-40e1-4c49-8845-32ce9c96cf28 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:13.927948 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 75/155] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:13.936440 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df9c0c-40e1-4c49-8845-32ce9c96cf28 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:14.029244 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:14.153307 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df9c0c-40e1-4c49-8845-32ce9c96cf28 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:14.160574 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e63ff2-53e2-4465-95ab-9058db2a1bda None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:14.223389 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae02259-bbfe-4b97-9eae-e8b0af80371b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:14.224511 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae02259-bbfe-4b97-9eae-e8b0af80371b None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:14.225080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae02259-bbfe-4b97-9eae-e8b0af80371b None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:14.225354 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae02259-bbfe-4b97-9eae-e8b0af80371b None admin] PATH_INFO: `/v3/users/b308731f39014d7098d27d547158b386` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:14.227541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae02259-bbfe-4b97-9eae-e8b0af80371b None admin] RBAC: Authorizing `identity:get_user(user_id=b308731f39014d7098d27d547158b386)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:14.229765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae02259-bbfe-4b97-9eae-e8b0af80371b None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:14.231430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 76/156] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:13 2022] GET /identity/v3/users/b308731f39014d7098d27d547158b386 => generated 247 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:14.234010 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12569201-5a3e-4c67-abc8-b8ccf8ec1c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:14.234678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12569201-5a3e-4c67-abc8-b8ccf8ec1c1d None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:14.234953 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12569201-5a3e-4c67-abc8-b8ccf8ec1c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:14.235276 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12569201-5a3e-4c67-abc8-b8ccf8ec1c1d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:14.246447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae02259-bbfe-4b97-9eae-e8b0af80371b None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:14.261135 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12569201-5a3e-4c67-abc8-b8ccf8ec1c1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:14.266902 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:14.269676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-12569201-5a3e-4c67-abc8-b8ccf8ec1c1d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:14.269676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:14.269676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:14.269676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:14.269676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:14.269676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:14.269676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:14.269676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:14.269676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:14.269676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:14.269676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:14.269676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:14.269676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:14.269676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:14.269676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:14.269676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:14.270628 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:14.270628 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:14.270628 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:14.270628 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:14.270628 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:14.270628 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:14.270628 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:14.270628 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:14.270628 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:14.270628 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:14.270628 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:14.270628 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:14.270628 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:14.270628 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:14.270628 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:14.270628 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:14.271451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:14.271451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:14.271451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:14.271451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:14.271451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:14.271451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:14.271451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:14.271451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:14.271451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:14.271451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:14.271451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:14.271451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:14.274476 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 81/157] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Aug 8 16:31:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:31:14.276236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f4a52-4235-445b-a327-346a5e7da9fa None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:14.276512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f4a52-4235-445b-a327-346a5e7da9fa None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:14.281216 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f4a52-4235-445b-a327-346a5e7da9fa None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:14.281216 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 82/158] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:14.281593 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f4a52-4235-445b-a327-346a5e7da9fa None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:14.406397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae02259-bbfe-4b97-9eae-e8b0af80371b None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:14.431902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f4a52-4235-445b-a327-346a5e7da9fa None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:14.465467 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:14.473504 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94de131-7e51-4fcb-ad53-daf278c770cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:14.474533 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94de131-7e51-4fcb-ad53-daf278c770cc None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:14.474533 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94de131-7e51-4fcb-ad53-daf278c770cc None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:14.474814 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94de131-7e51-4fcb-ad53-daf278c770cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:14.497147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ccb71-dc8e-43e6-b67a-6a36eb3c3494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:14.497761 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ccb71-dc8e-43e6-b67a-6a36eb3c3494 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:14.498034 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ccb71-dc8e-43e6-b67a-6a36eb3c3494 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:14.498302 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ccb71-dc8e-43e6-b67a-6a36eb3c3494 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:14.500921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ccb71-dc8e-43e6-b67a-6a36eb3c3494 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:14.501235 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ccb71-dc8e-43e6-b67a-6a36eb3c3494 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:14.502819 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ccb71-dc8e-43e6-b67a-6a36eb3c3494 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:14.515346 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 83/159] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:31:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:14.516814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b826a-aca2-4e04-826f-107404f0f21f None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:14.517328 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b826a-aca2-4e04-826f-107404f0f21f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:14.517700 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b826a-aca2-4e04-826f-107404f0f21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:14.542829 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-874b826a-aca2-4e04-826f-107404f0f21f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:14.544087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-874b826a-aca2-4e04-826f-107404f0f21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:14.605316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94de131-7e51-4fcb-ad53-daf278c770cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:14.606820 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94de131-7e51-4fcb-ad53-daf278c770cc None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:14.651920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:14.671588 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 84/160] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:14.671918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e00510-31d6-452f-8796-bbaf7b21824e None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:14.671918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e00510-31d6-452f-8796-bbaf7b21824e None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:14.672277 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e00510-31d6-452f-8796-bbaf7b21824e None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:14.673446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 85/161] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:14.674223 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e00510-31d6-452f-8796-bbaf7b21824e None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:14.800610 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 77/162] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:14.803071 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3fe37-cc92-4812-9277-4a433defcc07 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:14.803358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3fe37-cc92-4812-9277-4a433defcc07 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:14.803627 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3fe37-cc92-4812-9277-4a433defcc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:14.828052 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-8ce3fe37-cc92-4812-9277-4a433defcc07 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:14.829263 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-8ce3fe37-cc92-4812-9277-4a433defcc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:14.901991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 78/163] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:14.902838 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e00510-31d6-452f-8796-bbaf7b21824e None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:14.903125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3fe37-cc92-4812-9277-4a433defcc07 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:14.929564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:14.950748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:15.007293 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc6480-6e69-45f5-a539-7217ef2ee641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:15.007937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc6480-6e69-45f5-a539-7217ef2ee641 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:15.008209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc6480-6e69-45f5-a539-7217ef2ee641 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:15.008473 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc6480-6e69-45f5-a539-7217ef2ee641 None admin] PATH_INFO: `/v3/users/db4e226a05124c368e8ad70f2a45b7ad` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:15.011818 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3fe37-cc92-4812-9277-4a433defcc07 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:15.015322 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc6480-6e69-45f5-a539-7217ef2ee641 None admin] RBAC: Authorizing `identity:get_user(user_id=db4e226a05124c368e8ad70f2a45b7ad)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:15.016791 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc6480-6e69-45f5-a539-7217ef2ee641 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:15.018628 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 86/164] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:14 2022] GET /identity/v3/users/db4e226a05124c368e8ad70f2a45b7ad => generated 275 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:15.019236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc6480-6e69-45f5-a539-7217ef2ee641 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:15.049458 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29946ac-1945-4ceb-941c-1ad17eacbf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:15.050157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29946ac-1945-4ceb-941c-1ad17eacbf51 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:15.050436 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29946ac-1945-4ceb-941c-1ad17eacbf51 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:15.050732 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29946ac-1945-4ceb-941c-1ad17eacbf51 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:15.062815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29946ac-1945-4ceb-941c-1ad17eacbf51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:15.067734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-e29946ac-1945-4ceb-941c-1ad17eacbf51 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:15.067734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:15.067734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:15.067734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:15.067734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:15.067734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:15.067734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:15.067734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:15.067734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:15.067734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:15.067734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:15.067734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:15.067734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:15.067734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:15.067734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:15.067734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:15.068719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:15.068719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:15.068719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:15.068719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:15.068719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:15.068719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:15.068719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:15.068719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:15.068719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:15.068719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:15.068719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:15.068719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:15.068719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:15.068719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:15.068719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:15.068719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:15.069523 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:15.069523 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:15.069523 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:15.069523 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:15.069523 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:15.069523 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:15.069523 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:15.069523 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:15.069523 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:15.069523 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:15.069523 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:15.069523 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:15.069523 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 79/165] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:15.070959 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29946ac-1945-4ceb-941c-1ad17eacbf51 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:15.180420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29946ac-1945-4ceb-941c-1ad17eacbf51 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:15.182115 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:15.191691 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc6480-6e69-45f5-a539-7217ef2ee641 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:15.219934 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83848a4c-8407-41a1-aa36-00da1a315833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:15.220629 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83848a4c-8407-41a1-aa36-00da1a315833 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:15.220905 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83848a4c-8407-41a1-aa36-00da1a315833 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:15.221242 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83848a4c-8407-41a1-aa36-00da1a315833 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:15.229824 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c452b3-e2aa-4ba7-b4c3-b86cfdef0adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:15.230910 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c452b3-e2aa-4ba7-b4c3-b86cfdef0adc None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:15.231393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c452b3-e2aa-4ba7-b4c3-b86cfdef0adc None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:15.231921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c452b3-e2aa-4ba7-b4c3-b86cfdef0adc None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:15.233314 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c452b3-e2aa-4ba7-b4c3-b86cfdef0adc None admin] RBAC: Authorizing `identity:get_project(project_id=88933471b063476fb90fe841b4e9b75b)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:15.234285 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83848a4c-8407-41a1-aa36-00da1a315833 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:15.235267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c452b3-e2aa-4ba7-b4c3-b86cfdef0adc None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:15.237440 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 87/166] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:15 2022] GET /identity/v3/projects/88933471b063476fb90fe841b4e9b75b => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:15.238439 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c452b3-e2aa-4ba7-b4c3-b86cfdef0adc None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:15.243406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-83848a4c-8407-41a1-aa36-00da1a315833 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:31:15.243406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:15.243406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:15.243406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:15.243406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:15.243406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:15.243406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:15.243406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:15.243406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:15.243406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:15.243406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:15.243406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:15.243406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:15.243406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:15.243406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:15.243406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:15.244314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:15.244314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:15.244314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:15.244314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:15.244314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:15.244314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:15.244314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:15.244314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:15.244314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:15.244314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:15.244314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:15.244314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:15.244314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:15.244314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:15.244314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:15.244314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:15.246008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:15.246008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:15.246008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:15.246008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:15.246008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:15.246008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:15.246008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:15.246008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:15.246008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:15.246008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:15.246008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:31:15.246008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:15.246008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 80/167] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:15.246008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83848a4c-8407-41a1-aa36-00da1a315833 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:15.343206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:15.415217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83848a4c-8407-41a1-aa36-00da1a315833 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:15.424797 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c452b3-e2aa-4ba7-b4c3-b86cfdef0adc None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:15.461442 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a5cec-5e0b-4e86-83a5-ee35ed29bf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:15.461969 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a5cec-5e0b-4e86-83a5-ee35ed29bf95 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:15.462293 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a5cec-5e0b-4e86-83a5-ee35ed29bf95 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:15.462583 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a5cec-5e0b-4e86-83a5-ee35ed29bf95 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:15.463104 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ebba9-2d24-43c9-8537-4f22e47c54c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:15.463804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988a5cec-5e0b-4e86-83a5-ee35ed29bf95 None admin] RBAC: Adding query filter params (name=member) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:15.464068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988a5cec-5e0b-4e86-83a5-ee35ed29bf95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:15.464856 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ebba9-2d24-43c9-8537-4f22e47c54c2 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:15.465333 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ebba9-2d24-43c9-8537-4f22e47c54c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:15.465557 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988a5cec-5e0b-4e86-83a5-ee35ed29bf95 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:15.466011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ebba9-2d24-43c9-8537-4f22e47c54c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:15.481244 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 81/168] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:15.482688 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a5cec-5e0b-4e86-83a5-ee35ed29bf95 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:15.519633 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:15.547556 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ebba9-2d24-43c9-8537-4f22e47c54c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:15.549480 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ebba9-2d24-43c9-8537-4f22e47c54c2 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:15.599135 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:15.609059 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a5cec-5e0b-4e86-83a5-ee35ed29bf95 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:15.646277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a82f36-cd19-4f9b-81dd-fbb4ba0fcd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:15.646910 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a82f36-cd19-4f9b-81dd-fbb4ba0fcd44 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:15.647307 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a82f36-cd19-4f9b-81dd-fbb4ba0fcd44 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:15.647579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a82f36-cd19-4f9b-81dd-fbb4ba0fcd44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:15.648425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a82f36-cd19-4f9b-81dd-fbb4ba0fcd44 None admin] RBAC: Adding query filter params (role.id=e578ef831127463d9b48cc89fe20329c, scope.project.id=88933471b063476fb90fe841b4e9b75b, user.id=b308731f39014d7098d27d547158b386) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:15.648766 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a82f36-cd19-4f9b-81dd-fbb4ba0fcd44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:15.650304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a82f36-cd19-4f9b-81dd-fbb4ba0fcd44 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:15.662254 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 82/169] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:31:15 2022] GET /identity/v3/role_assignments?user.id=b308731f39014d7098d27d547158b386&scope.project.id=88933471b063476fb90fe841b4e9b75b&role.id=e578ef831127463d9b48cc89fe20329c => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:15.662254 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a82f36-cd19-4f9b-81dd-fbb4ba0fcd44 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:15.662501 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 88/170] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:15.662501 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ad3dc-f54e-4ee9-b018-d568cb6a6c1e None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:15.662501 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ad3dc-f54e-4ee9-b018-d568cb6a6c1e None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:15.662501 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ad3dc-f54e-4ee9-b018-d568cb6a6c1e None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:15.666088 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 89/171] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:15.669835 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ad3dc-f54e-4ee9-b018-d568cb6a6c1e None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:15.785944 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:15.804366 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a82f36-cd19-4f9b-81dd-fbb4ba0fcd44 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:15.809511 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ad3dc-f54e-4ee9-b018-d568cb6a6c1e None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:15.853204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36321946-d1cc-459b-a947-0424e56ad9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:15.853809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36321946-d1cc-459b-a947-0424e56ad9d2 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:15.855153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36321946-d1cc-459b-a947-0424e56ad9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:15.855153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36321946-d1cc-459b-a947-0424e56ad9d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:15.857544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36321946-d1cc-459b-a947-0424e56ad9d2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:15.857658 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f1bc8-6ccb-418a-8685-73a99e849237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:15.857766 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36321946-d1cc-459b-a947-0424e56ad9d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:15.857862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f1bc8-6ccb-418a-8685-73a99e849237 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:15.857862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f1bc8-6ccb-418a-8685-73a99e849237 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:15.857862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f1bc8-6ccb-418a-8685-73a99e849237 None admin] PATH_INFO: `/v3/users/db4e226a05124c368e8ad70f2a45b7ad` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:15.858053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36321946-d1cc-459b-a947-0424e56ad9d2 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:15.858120 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f1bc8-6ccb-418a-8685-73a99e849237 None admin] RBAC: Authorizing `identity:get_user(user_id=db4e226a05124c368e8ad70f2a45b7ad)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:15.860054 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f1bc8-6ccb-418a-8685-73a99e849237 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:15.861472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 90/172] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:15 2022] GET /identity/v3/users/db4e226a05124c368e8ad70f2a45b7ad => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:15.862734 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f522180-0fbb-44fe-b9de-15a40e9a18cd None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:15.863012 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f522180-0fbb-44fe-b9de-15a40e9a18cd None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:15.863276 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f522180-0fbb-44fe-b9de-15a40e9a18cd None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:15.864752 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 91/173] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:15.869389 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f522180-0fbb-44fe-b9de-15a40e9a18cd None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:15.876444 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 83/174] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:15.877711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544b28d-c279-47a6-b989-73ecb15aee29 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:15.877995 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544b28d-c279-47a6-b989-73ecb15aee29 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:15.878269 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544b28d-c279-47a6-b989-73ecb15aee29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:15.894804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:15.901771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-a544b28d-c279-47a6-b989-73ecb15aee29 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:15.903097 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-a544b28d-c279-47a6-b989-73ecb15aee29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:15.994071 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 84/175] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:15.995827 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544b28d-c279-47a6-b989-73ecb15aee29 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:16.000730 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f522180-0fbb-44fe-b9de-15a40e9a18cd None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:16.037446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9288080-e4cb-4bab-a72c-87aeff982b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:16.037988 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9288080-e4cb-4bab-a72c-87aeff982b06 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:16.038269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9288080-e4cb-4bab-a72c-87aeff982b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:16.038532 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9288080-e4cb-4bab-a72c-87aeff982b06 None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:16.039915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9288080-e4cb-4bab-a72c-87aeff982b06 None admin] RBAC: Authorizing `identity:get_project(project_id=88933471b063476fb90fe841b4e9b75b)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:16.041121 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9288080-e4cb-4bab-a72c-87aeff982b06 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:16.043925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 92/176] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:15 2022] GET /identity/v3/projects/88933471b063476fb90fe841b4e9b75b => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:16.045037 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3294a-1eae-4ffe-9289-69d0c51d929f None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:16.045312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3294a-1eae-4ffe-9289-69d0c51d929f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:16.045576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3294a-1eae-4ffe-9289-69d0c51d929f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:16.064149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-2ee3294a-1eae-4ffe-9289-69d0c51d929f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:16.065375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-2ee3294a-1eae-4ffe-9289-69d0c51d929f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:16.081547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:16.100530 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544b28d-c279-47a6-b989-73ecb15aee29 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:16.137301 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 93/177] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:16.137301 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3294a-1eae-4ffe-9289-69d0c51d929f None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:16.144114 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1702ba8-b980-4804-9001-8387b2941e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:16.144736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1702ba8-b980-4804-9001-8387b2941e9a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:16.145012 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1702ba8-b980-4804-9001-8387b2941e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:16.145286 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1702ba8-b980-4804-9001-8387b2941e9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:16.154587 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:16.213079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1702ba8-b980-4804-9001-8387b2941e9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:16.215331 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1702ba8-b980-4804-9001-8387b2941e9a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:16.240446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3294a-1eae-4ffe-9289-69d0c51d929f None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:16.274362 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8cd662-44c5-4099-8c46-21ebc1f47e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:16.275169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cd662-44c5-4099-8c46-21ebc1f47e3e None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:16.275529 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cd662-44c5-4099-8c46-21ebc1f47e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:16.276399 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cd662-44c5-4099-8c46-21ebc1f47e3e None admin] PATH_INFO: `/v3/users/1b6faeb56200440caf14f97686b3425b` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:16.277943 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8cd662-44c5-4099-8c46-21ebc1f47e3e None admin] RBAC: Authorizing `identity:get_user(user_id=1b6faeb56200440caf14f97686b3425b)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:16.279720 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8cd662-44c5-4099-8c46-21ebc1f47e3e None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:16.281483 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 94/178] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:16 2022] GET /identity/v3/users/1b6faeb56200440caf14f97686b3425b => generated 289 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:16.283047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14fac9-9225-424c-811a-1f75c088ebe8 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:16.283509 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14fac9-9225-424c-811a-1f75c088ebe8 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:16.283966 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14fac9-9225-424c-811a-1f75c088ebe8 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:16.285628 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 95/179] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:16.287109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14fac9-9225-424c-811a-1f75c088ebe8 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:16.298606 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:16.317757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 85/180] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:16.318605 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1702ba8-b980-4804-9001-8387b2941e9a None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:16.374085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:16.397904 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14fac9-9225-424c-811a-1f75c088ebe8 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:16.419818 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1702ba8-b980-4804-9001-8387b2941e9a None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:16.433437 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849d18c-9fde-4d47-92ba-a10e54593281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:16.434074 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849d18c-9fde-4d47-92ba-a10e54593281 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:16.434348 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849d18c-9fde-4d47-92ba-a10e54593281 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:16.434638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849d18c-9fde-4d47-92ba-a10e54593281 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:16.444556 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849d18c-9fde-4d47-92ba-a10e54593281 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:16.450254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-f849d18c-9fde-4d47-92ba-a10e54593281 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:31:16.450254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:16.450254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:16.450254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:16.450254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:16.450254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:16.450254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:16.450254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:16.450254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:16.450254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:16.450254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:16.450254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:16.450254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:16.450254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:16.450254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:16.450254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:16.451232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:16.451232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:16.451232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:16.451232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:16.451232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:16.451232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:16.451232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:16.451232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:16.451232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:16.451232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:16.451232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:16.451232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:16.451232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:16.451232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:16.451232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:16.451232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:16.452079 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:16.452079 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:16.452079 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:16.452079 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:16.452079 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:16.452079 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:16.452079 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:16.452079 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:16.452079 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:16.452079 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:16.452079 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:31:16.452079 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:16.452079 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 96/181] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:31:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:16.453301 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849d18c-9fde-4d47-92ba-a10e54593281 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:16.456472 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d3384-6438-4e10-a7e2-b0bd97c2ce8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:16.457075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d3384-6438-4e10-a7e2-b0bd97c2ce8a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:16.457343 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d3384-6438-4e10-a7e2-b0bd97c2ce8a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:16.460226 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d3384-6438-4e10-a7e2-b0bd97c2ce8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:16.460226 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8d3384-6438-4e10-a7e2-b0bd97c2ce8a None admin] RBAC: Adding query filter params (role.id=fd2599a3f408404599ecea94959efc4a, scope.project.id=88933471b063476fb90fe841b4e9b75b, user.id=db4e226a05124c368e8ad70f2a45b7ad) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:16.460226 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8d3384-6438-4e10-a7e2-b0bd97c2ce8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:16.460226 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8d3384-6438-4e10-a7e2-b0bd97c2ce8a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:16.466114 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 86/182] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:31:16 2022] GET /identity/v3/role_assignments?user.id=db4e226a05124c368e8ad70f2a45b7ad&scope.project.id=88933471b063476fb90fe841b4e9b75b&role.id=fd2599a3f408404599ecea94959efc4a => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:16.466766 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d3384-6438-4e10-a7e2-b0bd97c2ce8a None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:16.519945 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:16.579672 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849d18c-9fde-4d47-92ba-a10e54593281 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:16.581486 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d3384-6438-4e10-a7e2-b0bd97c2ce8a None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:16.621185 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895181b2-803a-4131-ada9-8b593d4a6afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:16.621944 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895181b2-803a-4131-ada9-8b593d4a6afe None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:16.622281 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895181b2-803a-4131-ada9-8b593d4a6afe None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:16.623433 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895181b2-803a-4131-ada9-8b593d4a6afe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:16.627611 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb8f9b-96b3-4fcf-b73c-d124231d8e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:16.628868 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb8f9b-96b3-4fcf-b73c-d124231d8e7b None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:16.628868 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb8f9b-96b3-4fcf-b73c-d124231d8e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:16.628868 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb8f9b-96b3-4fcf-b73c-d124231d8e7b None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:16.629874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb8f9b-96b3-4fcf-b73c-d124231d8e7b None admin] RBAC: Authorizing `identity:get_project(project_id=88933471b063476fb90fe841b4e9b75b)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:16.631506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb8f9b-96b3-4fcf-b73c-d124231d8e7b None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:16.633628 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 87/183] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:16 2022] GET /identity/v3/projects/88933471b063476fb90fe841b4e9b75b => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:16.634295 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb96406-91cb-4b6b-98da-4e82ae8a5f24 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:16.634930 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:16.635830 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb96406-91cb-4b6b-98da-4e82ae8a5f24 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:16.635830 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb96406-91cb-4b6b-98da-4e82ae8a5f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:16.639734 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895181b2-803a-4131-ada9-8b593d4a6afe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:16.650107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-895181b2-803a-4131-ada9-8b593d4a6afe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:16.650107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:16.650107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:16.650107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:16.650107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:16.650107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:16.650107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:16.650107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:16.650107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:16.650107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:16.650107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:16.650107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:16.650107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:16.650107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:16.650107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:16.650107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:16.651193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:16.651193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:16.651193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:16.651193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:16.651193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:16.651193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:16.651193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:16.651193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:16.651193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:16.651193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:16.651193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:16.651193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:16.651193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:16.651193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:16.651193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:16.651193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:16.652011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:16.652011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:16.652011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:16.652011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:16.652011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:16.652011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:16.652011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:16.652011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:16.652011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:16.652011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:16.652011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:16.652011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:16.652011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 97/184] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Aug 8 16:31:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:16.652011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895181b2-803a-4131-ada9-8b593d4a6afe None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:16.655257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-feb96406-91cb-4b6b-98da-4e82ae8a5f24 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:16.656508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-feb96406-91cb-4b6b-98da-4e82ae8a5f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:16.740393 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 88/185] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:16.741246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb96406-91cb-4b6b-98da-4e82ae8a5f24 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:16.760673 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895181b2-803a-4131-ada9-8b593d4a6afe None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:16.785263 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:16.794949 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fef9a-a36c-45ee-bdb5-c0cd32c7b3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:16.798871 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fef9a-a36c-45ee-bdb5-c0cd32c7b3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:16.798871 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fef9a-a36c-45ee-bdb5-c0cd32c7b3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:16.798871 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fef9a-a36c-45ee-bdb5-c0cd32c7b3a4 None admin] PATH_INFO: `/v3/projects/9fecbed0db964b66a7ddfa4d0f4df31c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:16.798871 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8fef9a-a36c-45ee-bdb5-c0cd32c7b3a4 None admin] RBAC: Authorizing `identity:get_project(project_id=9fecbed0db964b66a7ddfa4d0f4df31c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:16.798871 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8fef9a-a36c-45ee-bdb5-c0cd32c7b3a4 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:16.800445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 98/186] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:16 2022] GET /identity/v3/projects/9fecbed0db964b66a7ddfa4d0f4df31c => generated 312 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:16.801181 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fef9a-a36c-45ee-bdb5-c0cd32c7b3a4 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:16.844062 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb96406-91cb-4b6b-98da-4e82ae8a5f24 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:16.867648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:16.878897 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedcf245-e3aa-4016-8d34-8cecbb780c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:16.879505 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcf245-e3aa-4016-8d34-8cecbb780c93 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:16.879780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcf245-e3aa-4016-8d34-8cecbb780c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:16.880045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcf245-e3aa-4016-8d34-8cecbb780c93 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:16.880700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedcf245-e3aa-4016-8d34-8cecbb780c93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:16.880998 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedcf245-e3aa-4016-8d34-8cecbb780c93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:16.882379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedcf245-e3aa-4016-8d34-8cecbb780c93 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:16.890728 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 89/187] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:16.891529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedcf245-e3aa-4016-8d34-8cecbb780c93 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:16.905391 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fef9a-a36c-45ee-bdb5-c0cd32c7b3a4 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:16.941038 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191b80d-3e1d-448c-afa1-75584a55cf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:16.941747 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191b80d-3e1d-448c-afa1-75584a55cf5a None admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:16.941997 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191b80d-3e1d-448c-afa1-75584a55cf5a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:16.942268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191b80d-3e1d-448c-afa1-75584a55cf5a None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b/users/1b6faeb56200440caf14f97686b3425b/roles/94f7f84c31e0425495dcca9783e7a46e` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:16.944228 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191b80d-3e1d-448c-afa1-75584a55cf5a None admin] RBAC: Authorizing `identity:create_grant(project_id=88933471b063476fb90fe841b4e9b75b, user_id=1b6faeb56200440caf14f97686b3425b, role_id=94f7f84c31e0425495dcca9783e7a46e)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:16.945693 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191b80d-3e1d-448c-afa1-75584a55cf5a None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:16.995658 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedcf245-e3aa-4016-8d34-8cecbb780c93 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:16.997978 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:17.025309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 99/188] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:31:16 2022] PUT /identity/v3/projects/88933471b063476fb90fe841b4e9b75b/users/1b6faeb56200440caf14f97686b3425b/roles/94f7f84c31e0425495dcca9783e7a46e => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:31:17.026468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191b80d-3e1d-448c-afa1-75584a55cf5a None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:17.029408 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93b5ef-8e57-47bc-8e89-656ec0c1e3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:17.030032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93b5ef-8e57-47bc-8e89-656ec0c1e3da None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:17.030313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93b5ef-8e57-47bc-8e89-656ec0c1e3da None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:17.030606 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93b5ef-8e57-47bc-8e89-656ec0c1e3da None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:17.031284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93b5ef-8e57-47bc-8e89-656ec0c1e3da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:17.031588 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93b5ef-8e57-47bc-8e89-656ec0c1e3da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:17.033175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93b5ef-8e57-47bc-8e89-656ec0c1e3da None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:17.043094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 90/189] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:31:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:17.043895 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93b5ef-8e57-47bc-8e89-656ec0c1e3da None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:17.150373 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191b80d-3e1d-448c-afa1-75584a55cf5a None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:17.161890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93b5ef-8e57-47bc-8e89-656ec0c1e3da None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:17.189607 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:17.198255 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca172c2-5558-42dc-a565-b0cd96b7ca14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:17.198988 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca172c2-5558-42dc-a565-b0cd96b7ca14 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:17.199270 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca172c2-5558-42dc-a565-b0cd96b7ca14 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:17.199628 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca172c2-5558-42dc-a565-b0cd96b7ca14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:17.200871 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca172c2-5558-42dc-a565-b0cd96b7ca14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:17.202297 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca172c2-5558-42dc-a565-b0cd96b7ca14 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:17.204094 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 100/190] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:31:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:17.204667 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca172c2-5558-42dc-a565-b0cd96b7ca14 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:17.211199 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ae4a2-369a-4f78-9bdd-743d04b30da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:17.211877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ae4a2-369a-4f78-9bdd-743d04b30da3 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:17.212158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ae4a2-369a-4f78-9bdd-743d04b30da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:17.212425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ae4a2-369a-4f78-9bdd-743d04b30da3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:17.213158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ae4a2-369a-4f78-9bdd-743d04b30da3 None admin] RBAC: Adding query filter params (role.id=04bf40fcc26347de8167fc215f62c7aa, scope.project.id=9fecbed0db964b66a7ddfa4d0f4df31c, user.id=db4e226a05124c368e8ad70f2a45b7ad) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:17.214228 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ae4a2-369a-4f78-9bdd-743d04b30da3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:17.215701 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ae4a2-369a-4f78-9bdd-743d04b30da3 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:17.223450 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 91/191] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:31:17 2022] GET /identity/v3/role_assignments?user.id=db4e226a05124c368e8ad70f2a45b7ad&scope.project.id=9fecbed0db964b66a7ddfa4d0f4df31c&role.id=04bf40fcc26347de8167fc215f62c7aa => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:17.226433 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ae4a2-369a-4f78-9bdd-743d04b30da3 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:17.272980 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:17.320947 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca172c2-5558-42dc-a565-b0cd96b7ca14 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:17.363458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abacacc3-e545-4273-8048-623db92f4df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:17.364127 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abacacc3-e545-4273-8048-623db92f4df4 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:17.364403 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abacacc3-e545-4273-8048-623db92f4df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:17.364673 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abacacc3-e545-4273-8048-623db92f4df4 None admin] PATH_INFO: `/v3/users/0f5a82109d5348d2b94971c9bdb00c6d` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:17.378232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ae4a2-369a-4f78-9bdd-743d04b30da3 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:17.388925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abacacc3-e545-4273-8048-623db92f4df4 None admin] RBAC: Authorizing `identity:get_user(user_id=0f5a82109d5348d2b94971c9bdb00c6d)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:17.391426 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abacacc3-e545-4273-8048-623db92f4df4 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:17.395005 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 101/192] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:17 2022] GET /identity/v3/users/0f5a82109d5348d2b94971c9bdb00c6d => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:17.396747 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abacacc3-e545-4273-8048-623db92f4df4 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:17.429059 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21497f-b745-4008-a6fe-841c4cc3055d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:17.429619 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21497f-b745-4008-a6fe-841c4cc3055d None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:17.429893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21497f-b745-4008-a6fe-841c4cc3055d None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:17.430157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21497f-b745-4008-a6fe-841c4cc3055d None admin] PATH_INFO: `/v3/users/0f5a82109d5348d2b94971c9bdb00c6d` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:17.431356 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21497f-b745-4008-a6fe-841c4cc3055d None admin] RBAC: Authorizing `identity:get_user(user_id=0f5a82109d5348d2b94971c9bdb00c6d)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:17.432852 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21497f-b745-4008-a6fe-841c4cc3055d None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:17.434710 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 92/193] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:17 2022] GET /identity/v3/users/0f5a82109d5348d2b94971c9bdb00c6d => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:17.435465 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21497f-b745-4008-a6fe-841c4cc3055d None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:17.484924 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:17.540954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abacacc3-e545-4273-8048-623db92f4df4 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:17.567437 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:17.571240 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21497f-b745-4008-a6fe-841c4cc3055d None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:17.578108 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743249a1-dbd1-4eb6-8620-83eb67d69556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:17.578789 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743249a1-dbd1-4eb6-8620-83eb67d69556 None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:17.579068 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743249a1-dbd1-4eb6-8620-83eb67d69556 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:17.579663 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743249a1-dbd1-4eb6-8620-83eb67d69556 None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:17.582252 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743249a1-dbd1-4eb6-8620-83eb67d69556 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:17.582252 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743249a1-dbd1-4eb6-8620-83eb67d69556 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:17.608411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b11c6-7a82-4af5-9c9a-fe49adfec972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:17.609155 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b11c6-7a82-4af5-9c9a-fe49adfec972 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:17.609428 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b11c6-7a82-4af5-9c9a-fe49adfec972 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:17.609693 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b11c6-7a82-4af5-9c9a-fe49adfec972 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:17.675991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b11c6-7a82-4af5-9c9a-fe49adfec972 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:17.677492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b11c6-7a82-4af5-9c9a-fe49adfec972 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:17.709040 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 102/194] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Aug 8 16:31:17 2022] POST /identity/v3/users => generated 297 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:31:17.719775 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743249a1-dbd1-4eb6-8620-83eb67d69556 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:17.801853 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:17.806846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 93/195] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:17.814542 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b11c6-7a82-4af5-9c9a-fe49adfec972 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:17.869016 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743249a1-dbd1-4eb6-8620-83eb67d69556 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:17.882258 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:17.911848 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa74a6d-f6cf-4450-92c4-fea8df17f7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:17.912470 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa74a6d-f6cf-4450-92c4-fea8df17f7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:17.912783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa74a6d-f6cf-4450-92c4-fea8df17f7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:17.913017 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa74a6d-f6cf-4450-92c4-fea8df17f7c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:17.940130 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b11c6-7a82-4af5-9c9a-fe49adfec972 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:17.985270 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586b391-3605-4eba-8246-d426051da7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:17.985956 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586b391-3605-4eba-8246-d426051da7a7 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:17.986298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586b391-3605-4eba-8246-d426051da7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:17.986667 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586b391-3605-4eba-8246-d426051da7a7 None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:17.987886 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586b391-3605-4eba-8246-d426051da7a7 None admin] RBAC: Authorizing `identity:get_project(project_id=64dad222ddb64408b80bacb84185378f)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:17.989526 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586b391-3605-4eba-8246-d426051da7a7 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:17.991471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 94/196] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:17 2022] GET /identity/v3/projects/64dad222ddb64408b80bacb84185378f => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:17.992958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa74a6d-f6cf-4450-92c4-fea8df17f7c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:17.995206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa74a6d-f6cf-4450-92c4-fea8df17f7c2 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:17.998321 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586b391-3605-4eba-8246-d426051da7a7 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:18.103045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:18.142141 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586b391-3605-4eba-8246-d426051da7a7 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:18.142518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 103/197] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:18.148505 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa74a6d-f6cf-4450-92c4-fea8df17f7c2 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:18.161282 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:18.188567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288c94d-e9b7-4d00-9588-c38bfeade6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:18.189175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288c94d-e9b7-4d00-9588-c38bfeade6bc None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:18.189445 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288c94d-e9b7-4d00-9588-c38bfeade6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:18.189711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288c94d-e9b7-4d00-9588-c38bfeade6bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:18.191086 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4288c94d-e9b7-4d00-9588-c38bfeade6bc None admin] RBAC: Adding query filter params (role.id=e578ef831127463d9b48cc89fe20329c, scope.project.id=64dad222ddb64408b80bacb84185378f, user.id=0f5a82109d5348d2b94971c9bdb00c6d) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:18.191439 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4288c94d-e9b7-4d00-9588-c38bfeade6bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:18.192902 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4288c94d-e9b7-4d00-9588-c38bfeade6bc None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:18.201019 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 95/198] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:31:17 2022] GET /identity/v3/role_assignments?user.id=0f5a82109d5348d2b94971c9bdb00c6d&scope.project.id=64dad222ddb64408b80bacb84185378f&role.id=e578ef831127463d9b48cc89fe20329c => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:18.271941 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa74a6d-f6cf-4450-92c4-fea8df17f7c2 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:18.313141 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04014efe-1f24-4082-97f6-3a92c1315aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:18.313800 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04014efe-1f24-4082-97f6-3a92c1315aba None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:18.314078 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04014efe-1f24-4082-97f6-3a92c1315aba None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:18.314352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04014efe-1f24-4082-97f6-3a92c1315aba None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:18.315577 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04014efe-1f24-4082-97f6-3a92c1315aba None admin] RBAC: Authorizing `identity:get_project(project_id=64dad222ddb64408b80bacb84185378f)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:18.317109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04014efe-1f24-4082-97f6-3a92c1315aba None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:18.318914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 104/199] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:18 2022] GET /identity/v3/projects/64dad222ddb64408b80bacb84185378f => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:18.326118 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288c94d-e9b7-4d00-9588-c38bfeade6bc None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:18.441797 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288c94d-e9b7-4d00-9588-c38bfeade6bc None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:18.472203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:18.478325 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8037ee-5d2e-4f0f-9349-078c62412ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:18.479621 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8037ee-5d2e-4f0f-9349-078c62412ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:18.479907 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8037ee-5d2e-4f0f-9349-078c62412ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:18.480178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8037ee-5d2e-4f0f-9349-078c62412ee2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:18.480910 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8037ee-5d2e-4f0f-9349-078c62412ee2 None admin] RBAC: Adding query filter params (role.id=fd2599a3f408404599ecea94959efc4a, scope.project.id=64dad222ddb64408b80bacb84185378f, user.id=0f5a82109d5348d2b94971c9bdb00c6d) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:18.481214 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8037ee-5d2e-4f0f-9349-078c62412ee2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:18.482703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8037ee-5d2e-4f0f-9349-078c62412ee2 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:18.488686 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 96/200] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:31:18 2022] GET /identity/v3/role_assignments?user.id=0f5a82109d5348d2b94971c9bdb00c6d&scope.project.id=64dad222ddb64408b80bacb84185378f&role.id=fd2599a3f408404599ecea94959efc4a => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:19.393300 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241b4fd-fad4-4f9a-939d-77150f485569 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:19.393848 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241b4fd-fad4-4f9a-939d-77150f485569 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:19.393848 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241b4fd-fad4-4f9a-939d-77150f485569 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:19.395376 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 105/201] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:19.401497 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5946c7e-4e9a-4fa7-a0c5-9036c78f2442 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:19.402063 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5946c7e-4e9a-4fa7-a0c5-9036c78f2442 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:19.402063 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5946c7e-4e9a-4fa7-a0c5-9036c78f2442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:19.423637 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-f5946c7e-4e9a-4fa7-a0c5-9036c78f2442 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:19.424841 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-f5946c7e-4e9a-4fa7-a0c5-9036c78f2442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:19.520983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 97/202] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:31:19.568931 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241b4fd-fad4-4f9a-939d-77150f485569 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:19.672480 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:19.703346 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241b4fd-fad4-4f9a-939d-77150f485569 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:19.750546 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34281983-fceb-44e4-9a70-6199283e7dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:19.751218 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34281983-fceb-44e4-9a70-6199283e7dba None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:19.751497 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34281983-fceb-44e4-9a70-6199283e7dba None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:19.751775 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34281983-fceb-44e4-9a70-6199283e7dba None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:19.769122 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34281983-fceb-44e4-9a70-6199283e7dba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:19.774014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-34281983-fceb-44e4-9a70-6199283e7dba None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:31:19.774014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:19.774014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:19.774014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:19.774014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:19.774014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:19.774014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:19.774014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:19.774014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:19.774014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:19.774014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:19.774014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:19.774014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:19.774014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:19.774014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:19.774014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:19.776994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:19.776994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:19.776994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:19.776994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:19.776994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:19.776994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:19.776994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:19.776994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:19.776994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:19.776994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:19.776994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:19.776994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:19.776994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:19.776994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:19.776994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:19.776994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:19.779103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:19.779103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:19.779103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:19.779103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:19.779103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:19.779103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:19.779103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:19.779103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:19.779103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:19.779103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:19.779103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:31:19.779103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:19.779103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 106/203] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:19.786725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5946c7e-4e9a-4fa7-a0c5-9036c78f2442 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:19.929965 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5946c7e-4e9a-4fa7-a0c5-9036c78f2442 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:19.963501 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:19.977552 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f20ad-25c2-4eca-9fff-8ba291aa072a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:19.978172 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f20ad-25c2-4eca-9fff-8ba291aa072a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:19.978449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f20ad-25c2-4eca-9fff-8ba291aa072a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:19.978765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f20ad-25c2-4eca-9fff-8ba291aa072a None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:19.979689 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284f20ad-25c2-4eca-9fff-8ba291aa072a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:19.981139 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284f20ad-25c2-4eca-9fff-8ba291aa072a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:19.981340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284f20ad-25c2-4eca-9fff-8ba291aa072a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:19.994369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 98/204] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:20.003729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34281983-fceb-44e4-9a70-6199283e7dba None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:20.157321 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34281983-fceb-44e4-9a70-6199283e7dba None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:20.200238 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d1b835-ac3d-43e8-807c-53763ba94738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:20.200797 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1b835-ac3d-43e8-807c-53763ba94738 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:20.201074 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1b835-ac3d-43e8-807c-53763ba94738 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:20.201340 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1b835-ac3d-43e8-807c-53763ba94738 None admin] PATH_INFO: `/v3/users/1b6faeb56200440caf14f97686b3425b` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:20.202513 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d1b835-ac3d-43e8-807c-53763ba94738 None admin] RBAC: Authorizing `identity:get_user(user_id=1b6faeb56200440caf14f97686b3425b)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:20.203968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d1b835-ac3d-43e8-807c-53763ba94738 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:20.205713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 107/205] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:20 2022] GET /identity/v3/users/1b6faeb56200440caf14f97686b3425b => generated 289 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:20.211615 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f20ad-25c2-4eca-9fff-8ba291aa072a None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:20.357331 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f20ad-25c2-4eca-9fff-8ba291aa072a None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:20.404191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de930d04-5b45-4a99-a05c-a6814d37f542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:20.405121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de930d04-5b45-4a99-a05c-a6814d37f542 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:20.405121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de930d04-5b45-4a99-a05c-a6814d37f542 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:20.405415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de930d04-5b45-4a99-a05c-a6814d37f542 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:20.482497 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:20.502164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de930d04-5b45-4a99-a05c-a6814d37f542 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:20.503772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de930d04-5b45-4a99-a05c-a6814d37f542 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:20.549744 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465001d1-ccef-455c-baea-c18156808122 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:20.550268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465001d1-ccef-455c-baea-c18156808122 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:20.550268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465001d1-ccef-455c-baea-c18156808122 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:20.551644 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 108/206] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:20.558751 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c0839-8e39-4685-bd00-2bed1c59a6a4 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:20.559033 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c0839-8e39-4685-bd00-2bed1c59a6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:20.559307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c0839-8e39-4685-bd00-2bed1c59a6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:20.578921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:20.581497 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-7f5c0839-8e39-4685-bd00-2bed1c59a6a4 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:20.582717 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-7f5c0839-8e39-4685-bd00-2bed1c59a6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:20.674298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 99/207] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:20.675646 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815caae-4631-42f3-9991-ffd16cc87381 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:20.675922 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815caae-4631-42f3-9991-ffd16cc87381 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:20.676184 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815caae-4631-42f3-9991-ffd16cc87381 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:20.677534 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 100/208] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:20.678225 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815caae-4631-42f3-9991-ffd16cc87381 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:20.686537 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 109/209] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:31:20.687951 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f3943-ed2b-4bec-9a8f-e7902a510fbe None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:20.688242 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f3943-ed2b-4bec-9a8f-e7902a510fbe None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:20.688512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f3943-ed2b-4bec-9a8f-e7902a510fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:20.707600 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-cd8f3943-ed2b-4bec-9a8f-e7902a510fbe None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:20.708799 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-cd8f3943-ed2b-4bec-9a8f-e7902a510fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:20.781443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 110/210] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:20.783015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:20.787490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f3943-ed2b-4bec-9a8f-e7902a510fbe None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:20.803956 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815caae-4631-42f3-9991-ffd16cc87381 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:20.855163 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd89212f-26ea-4773-aab1-efa6ac161bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:20.856197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89212f-26ea-4773-aab1-efa6ac161bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:20.856676 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89212f-26ea-4773-aab1-efa6ac161bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:20.857161 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89212f-26ea-4773-aab1-efa6ac161bf8 None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:20.858545 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd89212f-26ea-4773-aab1-efa6ac161bf8 None admin] RBAC: Authorizing `identity:get_project(project_id=88933471b063476fb90fe841b4e9b75b)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:20.860820 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd89212f-26ea-4773-aab1-efa6ac161bf8 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:20.863906 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 101/211] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:20 2022] GET /identity/v3/projects/88933471b063476fb90fe841b4e9b75b => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:20.864920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd89212f-26ea-4773-aab1-efa6ac161bf8 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:20.887375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:20.931890 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f3943-ed2b-4bec-9a8f-e7902a510fbe None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:20.968961 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2fc033-238b-401d-a076-b7f7973d185d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:20.969600 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fc033-238b-401d-a076-b7f7973d185d None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:20.969876 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fc033-238b-401d-a076-b7f7973d185d None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:20.970142 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fc033-238b-401d-a076-b7f7973d185d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:20.979791 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd89212f-26ea-4773-aab1-efa6ac161bf8 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:20.980608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2fc033-238b-401d-a076-b7f7973d185d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:20.985253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-dc2fc033-238b-401d-a076-b7f7973d185d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:20.985253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:20.985253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:20.985253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:20.985253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:20.985253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:20.985253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:20.985253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:20.985253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:20.985253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:20.985253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:20.985253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:20.985253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:20.985253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:20.985253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:20.985253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:20.986251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:20.986251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:20.986251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:20.986251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:20.986251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:20.986251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:20.986251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:20.986251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:20.986251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:20.986251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:20.986251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:20.986251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:20.986251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:20.986251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:20.986251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:20.986251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:20.987105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:20.987105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:20.987105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:20.987105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:20.987105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:20.987105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:20.987105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:20.987105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:20.987105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:20.987105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:20.987105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:20.987105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:20.987105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 111/212] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:20.987775 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2fc033-238b-401d-a076-b7f7973d185d None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:21.016595 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531f864-3d33-4ea7-b735-430e2b99c4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:21.017223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531f864-3d33-4ea7-b735-430e2b99c4ab None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:21.017496 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531f864-3d33-4ea7-b735-430e2b99c4ab None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:21.017760 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531f864-3d33-4ea7-b735-430e2b99c4ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:21.018896 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531f864-3d33-4ea7-b735-430e2b99c4ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:21.020371 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531f864-3d33-4ea7-b735-430e2b99c4ab None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:21.021895 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 102/213] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:31:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:21.023027 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab7971-ff12-4162-afba-ca4157b3a54c None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:21.023309 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab7971-ff12-4162-afba-ca4157b3a54c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:21.023581 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab7971-ff12-4162-afba-ca4157b3a54c None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:21.028780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 103/214] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:21.028780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410f96c-fd33-4ab3-8944-3f1ccae5da36 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:21.028780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410f96c-fd33-4ab3-8944-3f1ccae5da36 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:21.028780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410f96c-fd33-4ab3-8944-3f1ccae5da36 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:21.028780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 104/215] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:21.028780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410f96c-fd33-4ab3-8944-3f1ccae5da36 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:21.052384 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:21.094215 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2fc033-238b-401d-a076-b7f7973d185d None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:21.120664 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:21.129464 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60137a4b-9b89-48f3-a3ff-fd2cd6ce62a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:21.130073 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60137a4b-9b89-48f3-a3ff-fd2cd6ce62a9 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:21.130348 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60137a4b-9b89-48f3-a3ff-fd2cd6ce62a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:21.130646 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60137a4b-9b89-48f3-a3ff-fd2cd6ce62a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:21.131371 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60137a4b-9b89-48f3-a3ff-fd2cd6ce62a9 None admin] RBAC: Adding query filter params (role.id=94f7f84c31e0425495dcca9783e7a46e, scope.project.id=88933471b063476fb90fe841b4e9b75b, user.id=1b6faeb56200440caf14f97686b3425b) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:21.131670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60137a4b-9b89-48f3-a3ff-fd2cd6ce62a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:21.132368 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410f96c-fd33-4ab3-8944-3f1ccae5da36 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:21.133107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60137a4b-9b89-48f3-a3ff-fd2cd6ce62a9 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:21.139045 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 112/216] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:31:20 2022] GET /identity/v3/role_assignments?user.id=1b6faeb56200440caf14f97686b3425b&scope.project.id=88933471b063476fb90fe841b4e9b75b&role.id=94f7f84c31e0425495dcca9783e7a46e => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:21.140908 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60137a4b-9b89-48f3-a3ff-fd2cd6ce62a9 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:21.172681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd10573-cbcb-4692-9679-649e5248bfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:21.173599 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd10573-cbcb-4692-9679-649e5248bfb4 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:21.174037 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd10573-cbcb-4692-9679-649e5248bfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:21.174302 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd10573-cbcb-4692-9679-649e5248bfb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:21.175045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd10573-cbcb-4692-9679-649e5248bfb4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:21.175348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd10573-cbcb-4692-9679-649e5248bfb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:21.176887 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd10573-cbcb-4692-9679-649e5248bfb4 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:21.187145 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 105/217] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:21.188518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40372d4-ea5b-4215-829d-41ce8d49412f None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:21.188802 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40372d4-ea5b-4215-829d-41ce8d49412f None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:21.189073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40372d4-ea5b-4215-829d-41ce8d49412f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:21.208119 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-a40372d4-ea5b-4215-829d-41ce8d49412f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:21.209329 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-a40372d4-ea5b-4215-829d-41ce8d49412f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:21.266346 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60137a4b-9b89-48f3-a3ff-fd2cd6ce62a9 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:21.290031 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 106/218] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:21.291244 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6fb12-c9fd-46a2-8e85-a56ff25979ed None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:21.292008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6fb12-c9fd-46a2-8e85-a56ff25979ed None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:21.292282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6fb12-c9fd-46a2-8e85-a56ff25979ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:21.307752 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b555c-01e0-4ecf-ad1f-bd8573a2dd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:21.308386 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b555c-01e0-4ecf-ad1f-bd8573a2dd2d None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:21.308765 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b555c-01e0-4ecf-ad1f-bd8573a2dd2d None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:21.309043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b555c-01e0-4ecf-ad1f-bd8573a2dd2d None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:21.309744 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b555c-01e0-4ecf-ad1f-bd8573a2dd2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:21.311222 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b555c-01e0-4ecf-ad1f-bd8573a2dd2d None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:21.311635 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-1ad6fb12-c9fd-46a2-8e85-a56ff25979ed None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:21.314226 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-1ad6fb12-c9fd-46a2-8e85-a56ff25979ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:21.323514 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:21.385883 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 107/219] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:21.386818 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6fb12-c9fd-46a2-8e85-a56ff25979ed None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:21.428791 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 113/220] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Aug 8 16:31:21 2022] POST /identity/v3/users => generated 297 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:31:21.429627 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b555c-01e0-4ecf-ad1f-bd8573a2dd2d None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:21.489070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:21.505403 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6fb12-c9fd-46a2-8e85-a56ff25979ed None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:21.544169 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698c7fc-237e-4f55-8ae9-eeb629cd6938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:21.544818 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698c7fc-237e-4f55-8ae9-eeb629cd6938 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:21.545092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698c7fc-237e-4f55-8ae9-eeb629cd6938 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:21.545363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698c7fc-237e-4f55-8ae9-eeb629cd6938 None admin] PATH_INFO: `/v3/users/fcdda2eb8bce465aab67d0d8a16d2615` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:21.554273 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b555c-01e0-4ecf-ad1f-bd8573a2dd2d None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:21.560462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698c7fc-237e-4f55-8ae9-eeb629cd6938 None admin] RBAC: Authorizing `identity:get_user(user_id=fcdda2eb8bce465aab67d0d8a16d2615)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:21.561880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698c7fc-237e-4f55-8ae9-eeb629cd6938 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:21.563939 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 108/221] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:21 2022] GET /identity/v3/users/fcdda2eb8bce465aab67d0d8a16d2615 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:21.564747 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698c7fc-237e-4f55-8ae9-eeb629cd6938 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:21.567533 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:21.607957 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a12e9-75ea-4ecb-b2fa-f5f9a13c3af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:21.608607 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a12e9-75ea-4ecb-b2fa-f5f9a13c3af0 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:21.608889 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a12e9-75ea-4ecb-b2fa-f5f9a13c3af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:21.609163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a12e9-75ea-4ecb-b2fa-f5f9a13c3af0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:21.620521 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a12e9-75ea-4ecb-b2fa-f5f9a13c3af0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:21.625689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-882a12e9-75ea-4ecb-b2fa-f5f9a13c3af0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:31:21.625689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:21.625689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:21.625689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:21.625689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:21.625689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:21.625689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:21.625689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:21.625689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:21.625689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:21.625689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:21.625689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:21.625689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:21.625689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:21.625689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:21.625689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:21.626772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:21.626772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:21.626772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:21.626772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:21.626772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:21.626772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:21.626772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:21.626772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:21.626772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:21.626772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:21.626772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:21.626772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:21.626772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:21.626772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:21.626772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:21.626772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:21.627639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:21.627639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:21.627639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:21.627639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:21.627639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:21.627639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:21.627639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:21.627639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:21.627639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:21.627639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:21.627639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:31:21.627639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:21.628265 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 114/222] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:31:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:21.628712 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a12e9-75ea-4ecb-b2fa-f5f9a13c3af0 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:21.676386 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698c7fc-237e-4f55-8ae9-eeb629cd6938 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:21.726474 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160853a8-9e19-4479-9cf4-38ab9ca2cd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:21.727427 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160853a8-9e19-4479-9cf4-38ab9ca2cd9f None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:21.727839 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160853a8-9e19-4479-9cf4-38ab9ca2cd9f None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:21.728203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160853a8-9e19-4479-9cf4-38ab9ca2cd9f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:21.742245 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160853a8-9e19-4479-9cf4-38ab9ca2cd9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:21.747838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-160853a8-9e19-4479-9cf4-38ab9ca2cd9f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:21.747838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:21.747838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:21.747838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:21.747838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:21.747838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:21.747838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:21.747838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:21.747838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:21.747838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:21.747838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:21.747838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:21.747838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:21.747838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:21.747838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:21.747838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:21.748981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:21.748981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:21.748981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:21.748981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:21.748981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:21.748981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:21.748981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:21.748981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:21.748981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:21.748981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:21.748981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:21.748981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:21.748981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:21.748981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:21.748981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:21.748981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:21.750001 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:21.750001 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:21.750001 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:21.750001 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:21.750001 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:21.750001 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:21.750001 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:21.750001 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:21.750001 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:21.750001 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:21.750001 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:21.750001 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:21.750850 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 109/223] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Aug 8 16:31:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:21.755946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160853a8-9e19-4479-9cf4-38ab9ca2cd9f None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:21.759133 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a12e9-75ea-4ecb-b2fa-f5f9a13c3af0 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:21.767800 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:21.801746 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b59e2-1f00-4edf-b870-43cdf76f3030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:21.802763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b59e2-1f00-4edf-b870-43cdf76f3030 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:21.803255 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b59e2-1f00-4edf-b870-43cdf76f3030 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:21.803672 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b59e2-1f00-4edf-b870-43cdf76f3030 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:21.832908 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:21.872754 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160853a8-9e19-4479-9cf4-38ab9ca2cd9f None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:21.896332 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b59e2-1f00-4edf-b870-43cdf76f3030 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:21.898848 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b59e2-1f00-4edf-b870-43cdf76f3030 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:21.913791 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53858a3-dc13-49c6-a928-2637570393ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:21.914506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53858a3-dc13-49c6-a928-2637570393ff None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:21.914881 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53858a3-dc13-49c6-a928-2637570393ff None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:21.915537 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53858a3-dc13-49c6-a928-2637570393ff None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:21.916672 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53858a3-dc13-49c6-a928-2637570393ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:21.917055 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53858a3-dc13-49c6-a928-2637570393ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:21.918918 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53858a3-dc13-49c6-a928-2637570393ff None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:21.934838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 110/224] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:21.936285 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53858a3-dc13-49c6-a928-2637570393ff None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:22.033971 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 115/225] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:22.035045 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b59e2-1f00-4edf-b870-43cdf76f3030 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:22.056854 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53858a3-dc13-49c6-a928-2637570393ff None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:22.075924 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:22.094189 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862d789-dfc7-47e8-9946-81ac97ad5fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:22.095053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862d789-dfc7-47e8-9946-81ac97ad5fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:22.095513 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862d789-dfc7-47e8-9946-81ac97ad5fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:22.095949 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862d789-dfc7-47e8-9946-81ac97ad5fc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:22.096824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3862d789-dfc7-47e8-9946-81ac97ad5fc2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:22.097260 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3862d789-dfc7-47e8-9946-81ac97ad5fc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:22.098894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3862d789-dfc7-47e8-9946-81ac97ad5fc2 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:22.107945 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 111/226] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:31:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:22.108454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:22.111316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862d789-dfc7-47e8-9946-81ac97ad5fc2 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:22.140584 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b59e2-1f00-4edf-b870-43cdf76f3030 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:22.180245 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2b34e-6a9e-434a-94e3-8bb0c2a6c4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:22.181127 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2b34e-6a9e-434a-94e3-8bb0c2a6c4ae None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:22.181512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2b34e-6a9e-434a-94e3-8bb0c2a6c4ae None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:22.181911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2b34e-6a9e-434a-94e3-8bb0c2a6c4ae None admin] PATH_INFO: `/v3/users/0f5a82109d5348d2b94971c9bdb00c6d` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:22.183943 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2b34e-6a9e-434a-94e3-8bb0c2a6c4ae None admin] RBAC: Authorizing `identity:get_user(user_id=0f5a82109d5348d2b94971c9bdb00c6d)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:22.185502 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2b34e-6a9e-434a-94e3-8bb0c2a6c4ae None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:22.187528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 116/227] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:22 2022] GET /identity/v3/users/0f5a82109d5348d2b94971c9bdb00c6d => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:22.188166 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2b34e-6a9e-434a-94e3-8bb0c2a6c4ae None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:22.244249 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862d789-dfc7-47e8-9946-81ac97ad5fc2 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:22.267744 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:22.284492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16736bad-f126-420e-96f6-2140d17be7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:22.285383 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16736bad-f126-420e-96f6-2140d17be7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:22.285811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16736bad-f126-420e-96f6-2140d17be7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:22.288428 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16736bad-f126-420e-96f6-2140d17be7c6 None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:22.288428 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16736bad-f126-420e-96f6-2140d17be7c6 None admin] RBAC: Authorizing `identity:get_project(project_id=64dad222ddb64408b80bacb84185378f)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:22.289198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16736bad-f126-420e-96f6-2140d17be7c6 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:22.292099 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 112/228] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:22 2022] GET /identity/v3/projects/64dad222ddb64408b80bacb84185378f => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:22.293047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16736bad-f126-420e-96f6-2140d17be7c6 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:22.301774 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2b34e-6a9e-434a-94e3-8bb0c2a6c4ae None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:22.344276 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a7015-b39b-463a-94f1-fe5da1b20892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:22.345158 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a7015-b39b-463a-94f1-fe5da1b20892 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:22.345672 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a7015-b39b-463a-94f1-fe5da1b20892 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:22.346043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a7015-b39b-463a-94f1-fe5da1b20892 None admin] PATH_INFO: `/v3/users/0f5a82109d5348d2b94971c9bdb00c6d` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:22.347661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a7015-b39b-463a-94f1-fe5da1b20892 None admin] RBAC: Authorizing `identity:get_user(user_id=0f5a82109d5348d2b94971c9bdb00c6d)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:22.349752 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a7015-b39b-463a-94f1-fe5da1b20892 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:22.352598 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 117/229] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:22 2022] GET /identity/v3/users/0f5a82109d5348d2b94971c9bdb00c6d => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:22.353637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a7015-b39b-463a-94f1-fe5da1b20892 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:22.355304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:22.404577 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16736bad-f126-420e-96f6-2140d17be7c6 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:22.458126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416e6af-988c-4300-a1a4-8775e21c54f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:22.459235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416e6af-988c-4300-a1a4-8775e21c54f4 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:22.459658 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416e6af-988c-4300-a1a4-8775e21c54f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:22.460596 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416e6af-988c-4300-a1a4-8775e21c54f4 None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:22.461881 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416e6af-988c-4300-a1a4-8775e21c54f4 None admin] RBAC: Authorizing `identity:get_project(project_id=64dad222ddb64408b80bacb84185378f)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:22.464684 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416e6af-988c-4300-a1a4-8775e21c54f4 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:22.467218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 113/230] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:22 2022] GET /identity/v3/projects/64dad222ddb64408b80bacb84185378f => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:22.468822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416e6af-988c-4300-a1a4-8775e21c54f4 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:22.478332 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a7015-b39b-463a-94f1-fe5da1b20892 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:22.517707 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a8df2-e380-47f0-9666-eb2a1d8ef624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:22.518312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a8df2-e380-47f0-9666-eb2a1d8ef624 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:22.518652 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a8df2-e380-47f0-9666-eb2a1d8ef624 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:22.518874 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a8df2-e380-47f0-9666-eb2a1d8ef624 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:22.519672 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a8df2-e380-47f0-9666-eb2a1d8ef624 None admin] RBAC: Adding query filter params (role.id=04bf40fcc26347de8167fc215f62c7aa, scope.project.id=64dad222ddb64408b80bacb84185378f, user.id=fcdda2eb8bce465aab67d0d8a16d2615) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:22.519976 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a8df2-e380-47f0-9666-eb2a1d8ef624 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:22.521409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a8df2-e380-47f0-9666-eb2a1d8ef624 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:22.534598 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 118/231] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:31:22 2022] GET /identity/v3/role_assignments?user.id=fcdda2eb8bce465aab67d0d8a16d2615&scope.project.id=64dad222ddb64408b80bacb84185378f&role.id=04bf40fcc26347de8167fc215f62c7aa => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:22.534598 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a8df2-e380-47f0-9666-eb2a1d8ef624 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:22.545929 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:22.657827 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416e6af-988c-4300-a1a4-8775e21c54f4 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:22.707270 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:22.732728 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4261f-eb77-4c69-9899-736d8f5a665a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:22.733435 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4261f-eb77-4c69-9899-736d8f5a665a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:22.733717 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4261f-eb77-4c69-9899-736d8f5a665a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:22.733988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4261f-eb77-4c69-9899-736d8f5a665a None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:22.736526 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a8df2-e380-47f0-9666-eb2a1d8ef624 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:22.738851 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4261f-eb77-4c69-9899-736d8f5a665a None admin] RBAC: Authorizing `identity:get_project(project_id=64dad222ddb64408b80bacb84185378f)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:22.740330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4261f-eb77-4c69-9899-736d8f5a665a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:22.742180 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 114/232] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:22 2022] GET /identity/v3/projects/64dad222ddb64408b80bacb84185378f => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:22.749839 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4261f-eb77-4c69-9899-736d8f5a665a None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:22.783001 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53415a51-9d54-493f-9464-8a3acfad237a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:22.784211 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53415a51-9d54-493f-9464-8a3acfad237a None admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:22.784996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53415a51-9d54-493f-9464-8a3acfad237a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:22.785728 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53415a51-9d54-493f-9464-8a3acfad237a None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f/users/0f5a82109d5348d2b94971c9bdb00c6d/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:22.788035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53415a51-9d54-493f-9464-8a3acfad237a None admin] RBAC: Authorizing `identity:create_grant(project_id=64dad222ddb64408b80bacb84185378f, user_id=0f5a82109d5348d2b94971c9bdb00c6d, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:22.789737 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53415a51-9d54-493f-9464-8a3acfad237a None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:22.875736 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 119/233] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:31:22 2022] PUT /identity/v3/projects/64dad222ddb64408b80bacb84185378f/users/0f5a82109d5348d2b94971c9bdb00c6d/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:31:22.884240 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4261f-eb77-4c69-9899-736d8f5a665a None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:22.934649 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60198cd2-fa42-4e2b-be30-79fa381862b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:22.934987 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60198cd2-fa42-4e2b-be30-79fa381862b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:22.938670 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60198cd2-fa42-4e2b-be30-79fa381862b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:22.938670 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60198cd2-fa42-4e2b-be30-79fa381862b2 None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f/users/0f5a82109d5348d2b94971c9bdb00c6d/roles/fd2599a3f408404599ecea94959efc4a` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:22.940432 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60198cd2-fa42-4e2b-be30-79fa381862b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=64dad222ddb64408b80bacb84185378f, user_id=0f5a82109d5348d2b94971c9bdb00c6d, role_id=fd2599a3f408404599ecea94959efc4a)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:22.942003 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60198cd2-fa42-4e2b-be30-79fa381862b2 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:23.037864 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 115/234] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:31:22 2022] PUT /identity/v3/projects/64dad222ddb64408b80bacb84185378f/users/0f5a82109d5348d2b94971c9bdb00c6d/roles/fd2599a3f408404599ecea94959efc4a => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:31:23.830953 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a3129-e59c-4b8a-aaa0-68c8752bad8e None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:23.831479 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a3129-e59c-4b8a-aaa0-68c8752bad8e None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:23.831479 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a3129-e59c-4b8a-aaa0-68c8752bad8e None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:23.832992 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 120/235] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:23.847544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c54f8-39b7-48c8-b6d1-36ffa165b271 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:23.848013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c54f8-39b7-48c8-b6d1-36ffa165b271 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:23.848088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c54f8-39b7-48c8-b6d1-36ffa165b271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:23.860417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285f815-4628-42c1-87a0-9cb4d1c8d509 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:23.860712 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285f815-4628-42c1-87a0-9cb4d1c8d509 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:23.860972 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285f815-4628-42c1-87a0-9cb4d1c8d509 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:23.862530 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 121/236] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:31:23.867216 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637e8c5-ac7f-49f3-aec9-5c42a77aa7aa None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:23.867496 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637e8c5-ac7f-49f3-aec9-5c42a77aa7aa None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:23.867771 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637e8c5-ac7f-49f3-aec9-5c42a77aa7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:23.875641 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-966c54f8-39b7-48c8-b6d1-36ffa165b271 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:23.876859 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-966c54f8-39b7-48c8-b6d1-36ffa165b271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:23.887777 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-b637e8c5-ac7f-49f3-aec9-5c42a77aa7aa None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:23.889120 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b637e8c5-ac7f-49f3-aec9-5c42a77aa7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:23.966759 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:24.001819 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:24.008159 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 116/237] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:24.018042 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 122/238] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:31:24.033474 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c54f8-39b7-48c8-b6d1-36ffa165b271 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:24.053224 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b637e8c5-ac7f-49f3-aec9-5c42a77aa7aa None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:24.218427 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c54f8-39b7-48c8-b6d1-36ffa165b271 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:24.255527 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b637e8c5-ac7f-49f3-aec9-5c42a77aa7aa None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:24.285520 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c5a31-849e-4215-b287-8def8d540090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:24.289122 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c5a31-849e-4215-b287-8def8d540090 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:24.289632 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c5a31-849e-4215-b287-8def8d540090 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:24.290101 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c5a31-849e-4215-b287-8def8d540090 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:24.307803 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c5a31-849e-4215-b287-8def8d540090 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:24.313223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-a14c5a31-849e-4215-b287-8def8d540090 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:31:24.313223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:24.313223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:24.313223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:24.313223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:24.313223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:24.313223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:24.313223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:24.313223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:24.313223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:24.313223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:24.313223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:24.313223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:24.313223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:24.313223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:24.313223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:24.314126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:24.314126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:24.314126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:24.314126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:24.314126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:24.314126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:24.314126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:24.314126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:24.314126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:24.314126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:24.314126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:24.314126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:24.314126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:24.314126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:24.314126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:24.314126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:24.314955 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:24.314955 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:24.314955 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:24.314955 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:24.314955 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:24.314955 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:24.314955 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:24.314955 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:24.314955 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:24.314955 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:24.314955 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:31:24.314955 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:24.319806 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f370f-6dc6-467f-9c16-c2b4bb878fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:24.321160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f370f-6dc6-467f-9c16-c2b4bb878fbe None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:24.321847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f370f-6dc6-467f-9c16-c2b4bb878fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:24.324486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f370f-6dc6-467f-9c16-c2b4bb878fbe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:24.325946 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f370f-6dc6-467f-9c16-c2b4bb878fbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:24.326452 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 117/239] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:24.328137 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f370f-6dc6-467f-9c16-c2b4bb878fbe None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:24.331080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c5a31-849e-4215-b287-8def8d540090 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:24.334553 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 123/240] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:31:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:24.336614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f370f-6dc6-467f-9c16-c2b4bb878fbe None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:24.410152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:24.451582 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:24.515070 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c5a31-849e-4215-b287-8def8d540090 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:24.517189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f370f-6dc6-467f-9c16-c2b4bb878fbe None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:24.575674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6caad2-6308-443a-861e-10d9a480dfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:24.576365 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6caad2-6308-443a-861e-10d9a480dfb9 None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:24.576643 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6caad2-6308-443a-861e-10d9a480dfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:24.576910 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6caad2-6308-443a-861e-10d9a480dfb9 None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:24.577570 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6caad2-6308-443a-861e-10d9a480dfb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:24.580387 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6caad2-6308-443a-861e-10d9a480dfb9 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:24.590927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ba612-8e03-447c-b276-0a8dad5733b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:24.591536 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ba612-8e03-447c-b276-0a8dad5733b6 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:24.591811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ba612-8e03-447c-b276-0a8dad5733b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:24.592079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ba612-8e03-447c-b276-0a8dad5733b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:24.592997 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ba612-8e03-447c-b276-0a8dad5733b6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:24.593296 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ba612-8e03-447c-b276-0a8dad5733b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:24.594804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ba612-8e03-447c-b276-0a8dad5733b6 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:24.610356 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 118/241] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:24.615980 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ba612-8e03-447c-b276-0a8dad5733b6 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:24.713844 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 124/242] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Aug 8 16:31:24 2022] POST /identity/v3/users => generated 293 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:31:24.741818 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ba612-8e03-447c-b276-0a8dad5733b6 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:24.743541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:24.790291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e622bf3-9653-4b2b-a190-25b91124636b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:24.790871 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e622bf3-9653-4b2b-a190-25b91124636b None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:24.791148 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e622bf3-9653-4b2b-a190-25b91124636b None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:24.792032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e622bf3-9653-4b2b-a190-25b91124636b None admin] PATH_INFO: `/v3/users/9f4b86e69f5b4c818485a62b75cb1fac` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:24.813933 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e622bf3-9653-4b2b-a190-25b91124636b None admin] RBAC: Authorizing `identity:get_user(user_id=9f4b86e69f5b4c818485a62b75cb1fac)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:24.815466 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e622bf3-9653-4b2b-a190-25b91124636b None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:24.817584 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 119/243] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:24 2022] GET /identity/v3/users/9f4b86e69f5b4c818485a62b75cb1fac => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:24.824089 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6caad2-6308-443a-861e-10d9a480dfb9 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:25.005310 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6caad2-6308-443a-861e-10d9a480dfb9 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:25.026193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:25.042053 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481a935-c71d-4b2c-b0bb-67b9e9483170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:25.042815 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481a935-c71d-4b2c-b0bb-67b9e9483170 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:25.043105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481a935-c71d-4b2c-b0bb-67b9e9483170 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:25.043379 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481a935-c71d-4b2c-b0bb-67b9e9483170 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:25.106953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039f522-7cc1-4d92-b442-7cd8ee93e65c None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:25.107467 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039f522-7cc1-4d92-b442-7cd8ee93e65c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:25.107467 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039f522-7cc1-4d92-b442-7cd8ee93e65c None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:25.109071 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 120/244] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:25.116671 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4ec91-33db-4891-9c96-65991cdf3b21 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:25.116962 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4ec91-33db-4891-9c96-65991cdf3b21 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:25.117227 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4ec91-33db-4891-9c96-65991cdf3b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:25.124220 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481a935-c71d-4b2c-b0bb-67b9e9483170 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:25.126009 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481a935-c71d-4b2c-b0bb-67b9e9483170 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:25.137931 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-17e4ec91-33db-4891-9c96-65991cdf3b21 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:25.139158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-17e4ec91-33db-4891-9c96-65991cdf3b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:25.211979 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 121/245] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:31:25.219915 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e131fea-bdbf-4fa2-8a1e-8add45d9e59b None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:25.220197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e131fea-bdbf-4fa2-8a1e-8add45d9e59b None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:25.220467 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e131fea-bdbf-4fa2-8a1e-8add45d9e59b None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:25.222939 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 122/246] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:25 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:25.229328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c0ce2-5012-4ea9-a4fc-748f8e1f53a0 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:25.230423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c0ce2-5012-4ea9-a4fc-748f8e1f53a0 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:25.230986 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c0ce2-5012-4ea9-a4fc-748f8e1f53a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:25.258477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-5c4c0ce2-5012-4ea9-a4fc-748f8e1f53a0 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:25.259272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 125/247] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:25.260180 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-5c4c0ce2-5012-4ea9-a4fc-748f8e1f53a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:25.260337 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481a935-c71d-4b2c-b0bb-67b9e9483170 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:25.296592 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:25.354784 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 123/248] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:25.355728 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c0ce2-5012-4ea9-a4fc-748f8e1f53a0 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:25.381717 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:25.430807 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481a935-c71d-4b2c-b0bb-67b9e9483170 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:25.474854 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d4257-2b38-429e-9bd2-03135f182f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:25.475487 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d4257-2b38-429e-9bd2-03135f182f92 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:25.475804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d4257-2b38-429e-9bd2-03135f182f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:25.476078 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d4257-2b38-429e-9bd2-03135f182f92 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:25.487795 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d4257-2b38-429e-9bd2-03135f182f92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:25.492767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-073d4257-2b38-429e-9bd2-03135f182f92 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:25.492767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:25.492767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:25.492767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:25.492767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:25.492767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:25.492767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:25.492767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:25.492767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:25.492767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:25.492767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:25.492767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:25.492767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:25.492767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:25.492767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:25.492767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:25.493729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:25.493729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:25.493729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:25.493729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:25.493729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:25.493729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:25.493729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:25.493729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:25.493729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:25.493729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:25.493729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:25.493729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:25.493729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:25.493729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:25.493729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:25.493729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:25.494605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:25.494605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:25.494605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:25.494605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:25.494605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:25.494605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:25.494605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:25.494605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:25.494605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:25.494605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:25.494605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:25.494605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:25.494605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 126/249] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:25.502601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d4257-2b38-429e-9bd2-03135f182f92 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:25.550847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c0ce2-5012-4ea9-a4fc-748f8e1f53a0 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:25.591435 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e4de6-517d-45b6-add1-d51f1ce2782d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:25.592070 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e4de6-517d-45b6-add1-d51f1ce2782d None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:25.592346 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e4de6-517d-45b6-add1-d51f1ce2782d None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:25.592650 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e4de6-517d-45b6-add1-d51f1ce2782d None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:25.594538 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e4de6-517d-45b6-add1-d51f1ce2782d None admin] RBAC: Authorizing `identity:get_project(project_id=64dad222ddb64408b80bacb84185378f)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:25.595998 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e4de6-517d-45b6-add1-d51f1ce2782d None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:25.597736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 124/250] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:25 2022] GET /identity/v3/projects/64dad222ddb64408b80bacb84185378f => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:25.598547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e4de6-517d-45b6-add1-d51f1ce2782d None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:25.685402 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:25.727197 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d4257-2b38-429e-9bd2-03135f182f92 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:25.737041 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:25.753156 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e4de6-517d-45b6-add1-d51f1ce2782d None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:25.769120 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03ba2a-facd-4f97-b6ac-3829d735a7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:25.769761 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03ba2a-facd-4f97-b6ac-3829d735a7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:25.770036 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03ba2a-facd-4f97-b6ac-3829d735a7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:25.770306 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03ba2a-facd-4f97-b6ac-3829d735a7e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:25.786110 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03ba2a-facd-4f97-b6ac-3829d735a7e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:25.790862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-bc03ba2a-facd-4f97-b6ac-3829d735a7e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:31:25.790862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:25.790862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:25.790862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:25.790862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:25.790862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:25.790862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:25.790862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:25.790862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:25.790862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:25.790862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:25.790862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:25.790862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:25.790862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:25.790862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:25.790862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:25.791882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:25.791882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:25.791882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:25.791882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:25.791882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:25.791882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:25.791882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:25.791882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:25.791882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:25.791882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:25.791882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:25.791882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:25.791882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:25.791882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:25.791882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:25.791882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:25.792718 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:25.792718 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:25.792718 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:25.792718 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:25.792718 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:25.792718 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:25.792718 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:25.792718 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:25.792718 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:25.792718 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:25.792718 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:31:25.792718 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:25.792718 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 127/251] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:31:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:25.793462 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03ba2a-facd-4f97-b6ac-3829d735a7e0 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:25.810665 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2d589-eef7-4d4a-a22e-db14feeea969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:25.811750 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2d589-eef7-4d4a-a22e-db14feeea969 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:25.812092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2d589-eef7-4d4a-a22e-db14feeea969 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:25.812696 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2d589-eef7-4d4a-a22e-db14feeea969 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:25.813424 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2d589-eef7-4d4a-a22e-db14feeea969 None admin] RBAC: Adding query filter params (name=member) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:25.814009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2d589-eef7-4d4a-a22e-db14feeea969 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:25.815874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2d589-eef7-4d4a-a22e-db14feeea969 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:25.828313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 125/252] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:25.830625 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1731b37-f791-4191-b89e-addd99a8f894 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:25.831126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1731b37-f791-4191-b89e-addd99a8f894 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:25.831582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1731b37-f791-4191-b89e-addd99a8f894 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:25.833784 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 126/253] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:25.834358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1731b37-f791-4191-b89e-addd99a8f894 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:25.917113 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03ba2a-facd-4f97-b6ac-3829d735a7e0 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:25.955613 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734c9c8-cb91-4166-9929-1375acebdf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:25.956249 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734c9c8-cb91-4166-9929-1375acebdf4e None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:25.956525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734c9c8-cb91-4166-9929-1375acebdf4e None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:25.956903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734c9c8-cb91-4166-9929-1375acebdf4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:25.957647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734c9c8-cb91-4166-9929-1375acebdf4e None admin] RBAC: Adding query filter params (role.id=94f7f84c31e0425495dcca9783e7a46e, scope.project.id=64dad222ddb64408b80bacb84185378f, user.id=9f4b86e69f5b4c818485a62b75cb1fac) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:25.958034 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734c9c8-cb91-4166-9929-1375acebdf4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:25.959506 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734c9c8-cb91-4166-9929-1375acebdf4e None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:25.961964 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1731b37-f791-4191-b89e-addd99a8f894 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:25.965819 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 128/254] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:31:25 2022] GET /identity/v3/role_assignments?user.id=9f4b86e69f5b4c818485a62b75cb1fac&scope.project.id=64dad222ddb64408b80bacb84185378f&role.id=94f7f84c31e0425495dcca9783e7a46e => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:25.966444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734c9c8-cb91-4166-9929-1375acebdf4e None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:25.970413 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:26.000934 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8926077-226d-403f-8fff-205ad2d1cc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:26.001939 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8926077-226d-403f-8fff-205ad2d1cc7c None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:26.002435 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8926077-226d-403f-8fff-205ad2d1cc7c None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:26.002945 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8926077-226d-403f-8fff-205ad2d1cc7c None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:26.003792 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8926077-226d-403f-8fff-205ad2d1cc7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:26.004267 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8926077-226d-403f-8fff-205ad2d1cc7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:26.005951 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8926077-226d-403f-8fff-205ad2d1cc7c None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:26.016177 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 127/255] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:26.017966 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9db43-dfed-4102-9d12-7f446689bdd1 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:26.018455 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9db43-dfed-4102-9d12-7f446689bdd1 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:26.019080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9db43-dfed-4102-9d12-7f446689bdd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:26.040142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-50e9db43-dfed-4102-9d12-7f446689bdd1 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:26.041906 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-50e9db43-dfed-4102-9d12-7f446689bdd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:26.050348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:26.089368 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734c9c8-cb91-4166-9929-1375acebdf4e None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:26.131727 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4b0f6b-ae83-4453-be3f-7a7151704b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:26.132382 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b0f6b-ae83-4453-be3f-7a7151704b30 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:26.132926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b0f6b-ae83-4453-be3f-7a7151704b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:26.133199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b0f6b-ae83-4453-be3f-7a7151704b30 None admin] PATH_INFO: `/v3/users/fcdda2eb8bce465aab67d0d8a16d2615` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:26.134469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4b0f6b-ae83-4453-be3f-7a7151704b30 None admin] RBAC: Authorizing `identity:get_user(user_id=fcdda2eb8bce465aab67d0d8a16d2615)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:26.137633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4b0f6b-ae83-4453-be3f-7a7151704b30 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:26.139659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 129/256] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:25 2022] GET /identity/v3/users/fcdda2eb8bce465aab67d0d8a16d2615 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:26.140251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4b0f6b-ae83-4453-be3f-7a7151704b30 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:26.193189 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 128/257] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:26.194048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9db43-dfed-4102-9d12-7f446689bdd1 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:26.221658 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:26.254868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4b0f6b-ae83-4453-be3f-7a7151704b30 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:26.291008 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72235c5-af4a-4b06-9811-6e4a033c3caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:26.291755 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72235c5-af4a-4b06-9811-6e4a033c3caf None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:26.292029 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72235c5-af4a-4b06-9811-6e4a033c3caf None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:26.292298 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72235c5-af4a-4b06-9811-6e4a033c3caf None admin] PATH_INFO: `/v3/users/0f5a82109d5348d2b94971c9bdb00c6d` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:26.293451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72235c5-af4a-4b06-9811-6e4a033c3caf None admin] RBAC: Authorizing `identity:get_user(user_id=0f5a82109d5348d2b94971c9bdb00c6d)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:26.294918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72235c5-af4a-4b06-9811-6e4a033c3caf None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:26.296849 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 130/258] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:26 2022] GET /identity/v3/users/0f5a82109d5348d2b94971c9bdb00c6d => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:26.297417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72235c5-af4a-4b06-9811-6e4a033c3caf None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:26.323691 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9db43-dfed-4102-9d12-7f446689bdd1 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:26.360922 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed56e9-8fd6-4b62-b2fb-001e9d059426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:26.361924 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed56e9-8fd6-4b62-b2fb-001e9d059426 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:26.362415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed56e9-8fd6-4b62-b2fb-001e9d059426 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:26.362918 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed56e9-8fd6-4b62-b2fb-001e9d059426 None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:26.364254 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed56e9-8fd6-4b62-b2fb-001e9d059426 None admin] RBAC: Authorizing `identity:get_project(project_id=64dad222ddb64408b80bacb84185378f)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:26.365897 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed56e9-8fd6-4b62-b2fb-001e9d059426 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:26.370911 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 129/259] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:26 2022] GET /identity/v3/projects/64dad222ddb64408b80bacb84185378f => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:26.372057 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed56e9-8fd6-4b62-b2fb-001e9d059426 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:26.394941 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:26.430997 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72235c5-af4a-4b06-9811-6e4a033c3caf None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:26.480870 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed56e9-8fd6-4b62-b2fb-001e9d059426 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:26.501383 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c7865-eb1f-4279-84c7-c27e6e81c0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:26.502045 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c7865-eb1f-4279-84c7-c27e6e81c0b5 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:26.502329 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c7865-eb1f-4279-84c7-c27e6e81c0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:26.502886 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c7865-eb1f-4279-84c7-c27e6e81c0b5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:26.520818 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde72131-1670-42b4-b99d-a0de038380a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:26.521439 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:26.522075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde72131-1670-42b4-b99d-a0de038380a3 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:26.522694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde72131-1670-42b4-b99d-a0de038380a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:26.523164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde72131-1670-42b4-b99d-a0de038380a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:26.526175 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c7865-eb1f-4279-84c7-c27e6e81c0b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:26.540780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-d09c7865-eb1f-4279-84c7-c27e6e81c0b5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:26.540780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:26.540780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:26.540780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:26.540780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:26.540780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:26.540780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:26.540780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:26.540780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:26.540780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:26.540780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:26.540780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:26.540780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:26.540780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:26.540780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:26.540780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:26.547591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:26.547591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:26.547591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:26.547591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:26.547591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:26.547591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:26.547591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:26.547591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:26.547591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:26.547591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:26.547591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:26.547591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:26.547591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:26.547591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:26.547591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:26.547591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:26.548401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:26.548401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:26.548401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:26.548401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:26.548401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:26.548401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:26.548401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:26.548401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:26.548401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:26.548401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:26.548401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:26.548401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:26.548401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 131/260] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Aug 8 16:31:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:26.548401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c7865-eb1f-4279-84c7-c27e6e81c0b5 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:26.608757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde72131-1670-42b4-b99d-a0de038380a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:26.610599 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde72131-1670-42b4-b99d-a0de038380a3 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:26.633287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:26.725084 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c7865-eb1f-4279-84c7-c27e6e81c0b5 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:26.791085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4086bc-a91e-4b44-b164-383510b824c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:26.792735 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4086bc-a91e-4b44-b164-383510b824c2 None admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:26.793221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4086bc-a91e-4b44-b164-383510b824c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:26.793696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4086bc-a91e-4b44-b164-383510b824c2 None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f/users/fcdda2eb8bce465aab67d0d8a16d2615/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:26.796744 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4086bc-a91e-4b44-b164-383510b824c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=64dad222ddb64408b80bacb84185378f, user_id=fcdda2eb8bce465aab67d0d8a16d2615, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:26.799583 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4086bc-a91e-4b44-b164-383510b824c2 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:26.809975 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 130/261] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:26.811134 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde72131-1670-42b4-b99d-a0de038380a3 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:26.886467 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 132/262] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:31:26 2022] PUT /identity/v3/projects/64dad222ddb64408b80bacb84185378f/users/fcdda2eb8bce465aab67d0d8a16d2615/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:31:26.887805 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4086bc-a91e-4b44-b164-383510b824c2 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:27.003240 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde72131-1670-42b4-b99d-a0de038380a3 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:27.027454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:27.058124 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:27.071352 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118d014-b2c7-4f60-a5a8-75a33f25d89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:27.072155 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118d014-b2c7-4f60-a5a8-75a33f25d89c None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:27.072605 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118d014-b2c7-4f60-a5a8-75a33f25d89c None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:27.073505 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118d014-b2c7-4f60-a5a8-75a33f25d89c None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:27.075346 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118d014-b2c7-4f60-a5a8-75a33f25d89c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:27.075767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118d014-b2c7-4f60-a5a8-75a33f25d89c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:27.077741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118d014-b2c7-4f60-a5a8-75a33f25d89c None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:27.096602 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 131/263] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:31:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:27.102965 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118d014-b2c7-4f60-a5a8-75a33f25d89c None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:27.196154 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4086bc-a91e-4b44-b164-383510b824c2 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:27.266604 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188ebe1-e2f5-450d-9937-ed285192dde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:27.267524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188ebe1-e2f5-450d-9937-ed285192dde6 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:27.267942 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188ebe1-e2f5-450d-9937-ed285192dde6 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:27.268358 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188ebe1-e2f5-450d-9937-ed285192dde6 None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:27.270665 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188ebe1-e2f5-450d-9937-ed285192dde6 None admin] RBAC: Authorizing `identity:get_project(project_id=64dad222ddb64408b80bacb84185378f)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:27.272281 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188ebe1-e2f5-450d-9937-ed285192dde6 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:27.274156 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 133/264] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:26 2022] GET /identity/v3/projects/64dad222ddb64408b80bacb84185378f => generated 302 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:27.280139 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188ebe1-e2f5-450d-9937-ed285192dde6 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:27.295042 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118d014-b2c7-4f60-a5a8-75a33f25d89c None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:27.366799 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cfed1-f2cf-47f6-9f07-077eee3f3cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:27.367681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cfed1-f2cf-47f6-9f07-077eee3f3cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:27.368125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cfed1-f2cf-47f6-9f07-077eee3f3cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:27.368537 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cfed1-f2cf-47f6-9f07-077eee3f3cc1 None admin] PATH_INFO: `/v3/users/0f5a82109d5348d2b94971c9bdb00c6d` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:27.371476 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91cfed1-f2cf-47f6-9f07-077eee3f3cc1 None admin] RBAC: Authorizing `identity:get_user(user_id=0f5a82109d5348d2b94971c9bdb00c6d)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:27.373084 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91cfed1-f2cf-47f6-9f07-077eee3f3cc1 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:27.374888 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 132/265] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:27 2022] GET /identity/v3/users/0f5a82109d5348d2b94971c9bdb00c6d => generated 283 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:27.380911 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cfed1-f2cf-47f6-9f07-077eee3f3cc1 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:27.444040 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188ebe1-e2f5-450d-9937-ed285192dde6 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:27.482403 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:27.485216 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:27.507804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd7c7e-b8e0-4ee5-89ed-22c262c25a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:27.508485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd7c7e-b8e0-4ee5-89ed-22c262c25a23 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:27.508836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd7c7e-b8e0-4ee5-89ed-22c262c25a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:27.509109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd7c7e-b8e0-4ee5-89ed-22c262c25a23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:27.509841 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd7c7e-b8e0-4ee5-89ed-22c262c25a23 None admin] RBAC: Adding query filter params (role.id=e578ef831127463d9b48cc89fe20329c, scope.project.id=64dad222ddb64408b80bacb84185378f, user.id=0f5a82109d5348d2b94971c9bdb00c6d) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:27.514994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd7c7e-b8e0-4ee5-89ed-22c262c25a23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:27.517842 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd7c7e-b8e0-4ee5-89ed-22c262c25a23 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:27.533835 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 134/266] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:31:27 2022] GET /identity/v3/role_assignments?user.id=0f5a82109d5348d2b94971c9bdb00c6d&scope.project.id=64dad222ddb64408b80bacb84185378f&role.id=e578ef831127463d9b48cc89fe20329c => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:27.586745 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cfed1-f2cf-47f6-9f07-077eee3f3cc1 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:27.648263 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc839e8-71a5-452b-ac43-eb2745811849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:27.648897 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc839e8-71a5-452b-ac43-eb2745811849 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:27.649177 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc839e8-71a5-452b-ac43-eb2745811849 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:27.649448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc839e8-71a5-452b-ac43-eb2745811849 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:27.750137 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc839e8-71a5-452b-ac43-eb2745811849 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:27.751586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc839e8-71a5-452b-ac43-eb2745811849 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:27.807194 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:27.831300 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a4c8a-5512-4fb9-8863-4ce44ec18397 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:27.831661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a4c8a-5512-4fb9-8863-4ce44ec18397 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:27.831860 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a4c8a-5512-4fb9-8863-4ce44ec18397 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:27.833300 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 135/267] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:27.834482 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d3202-35a6-4451-a430-a9056b9f0b4a None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:27.834765 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d3202-35a6-4451-a430-a9056b9f0b4a None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:27.835040 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d3202-35a6-4451-a430-a9056b9f0b4a None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:27.836658 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 136/268] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:27.839998 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ce427-4654-47ef-ae34-e9482586e2b3 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:27.840316 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ce427-4654-47ef-ae34-e9482586e2b3 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:27.840562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ce427-4654-47ef-ae34-e9482586e2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:27.869330 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-0e8ce427-4654-47ef-ae34-e9482586e2b3 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:27.870618 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-0e8ce427-4654-47ef-ae34-e9482586e2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:27.904924 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 133/269] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:27.906448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef03abf-be92-4c10-8840-0cdb15f781cb None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:27.906857 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef03abf-be92-4c10-8840-0cdb15f781cb None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:27.907160 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef03abf-be92-4c10-8840-0cdb15f781cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:27.935053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-cef03abf-be92-4c10-8840-0cdb15f781cb None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:27.936553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-cef03abf-be92-4c10-8840-0cdb15f781cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:27.965008 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 137/270] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:27.965862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ce427-4654-47ef-ae34-e9482586e2b3 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:28.043523 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 134/271] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:28.044299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef03abf-be92-4c10-8840-0cdb15f781cb None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:28.092342 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ce427-4654-47ef-ae34-e9482586e2b3 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:28.118220 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:28.132511 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74899b76-a91f-4aaf-9ba9-89d3147b9d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:28.133241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74899b76-a91f-4aaf-9ba9-89d3147b9d7a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:28.133588 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74899b76-a91f-4aaf-9ba9-89d3147b9d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:28.133928 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74899b76-a91f-4aaf-9ba9-89d3147b9d7a None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:28.135740 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74899b76-a91f-4aaf-9ba9-89d3147b9d7a None admin] RBAC: Authorizing `identity:get_project(project_id=64dad222ddb64408b80bacb84185378f)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:28.137510 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74899b76-a91f-4aaf-9ba9-89d3147b9d7a None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:28.139720 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 138/272] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:27 2022] GET /identity/v3/projects/64dad222ddb64408b80bacb84185378f => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:28.140500 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74899b76-a91f-4aaf-9ba9-89d3147b9d7a None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:28.149825 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:28.188163 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef03abf-be92-4c10-8840-0cdb15f781cb None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:28.228739 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ddf07-75ac-45b5-a8e9-4146c58803a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:28.230096 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ddf07-75ac-45b5-a8e9-4146c58803a7 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:28.230096 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ddf07-75ac-45b5-a8e9-4146c58803a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:28.230391 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ddf07-75ac-45b5-a8e9-4146c58803a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:28.231558 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ddf07-75ac-45b5-a8e9-4146c58803a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:28.233036 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ddf07-75ac-45b5-a8e9-4146c58803a7 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:28.234837 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 135/273] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:31:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:28.235454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ddf07-75ac-45b5-a8e9-4146c58803a7 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:28.281361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74899b76-a91f-4aaf-9ba9-89d3147b9d7a None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:28.320891 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbeb76-2612-4c25-a453-8772852e78e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:28.321858 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbeb76-2612-4c25-a453-8772852e78e9 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:28.323981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbeb76-2612-4c25-a453-8772852e78e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:28.324496 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbeb76-2612-4c25-a453-8772852e78e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:28.336605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffbeb76-2612-4c25-a453-8772852e78e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:28.343522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-5ffbeb76-2612-4c25-a453-8772852e78e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:28.343522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:28.343522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:28.343522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:28.343522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:28.343522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:28.343522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:28.343522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:28.343522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:28.343522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:28.343522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:28.343522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:28.343522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:28.343522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:28.343522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:28.343522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:28.344469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:28.344469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:28.344469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:28.344469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:28.344469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:28.344469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:28.344469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:28.344469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:28.344469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:28.344469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:28.344469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:28.344469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:28.344469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:28.344469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:28.344469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:28.344469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:28.345272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:28.345272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:28.345272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:28.345272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:28.345272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:28.345272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:28.345272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:28.345272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:28.345272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:28.345272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:28.345272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:28.345272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:28.348123 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 139/274] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:28.349198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbeb76-2612-4c25-a453-8772852e78e9 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:28.349725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ddf07-75ac-45b5-a8e9-4146c58803a7 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:28.386486 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740e08f-5c55-49d8-ae95-49ea42336656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:28.387120 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740e08f-5c55-49d8-ae95-49ea42336656 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:28.387425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740e08f-5c55-49d8-ae95-49ea42336656 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:28.387654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740e08f-5c55-49d8-ae95-49ea42336656 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:28.388380 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740e08f-5c55-49d8-ae95-49ea42336656 None admin] RBAC: Adding query filter params (role.id=fd2599a3f408404599ecea94959efc4a, scope.project.id=64dad222ddb64408b80bacb84185378f, user.id=0f5a82109d5348d2b94971c9bdb00c6d) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:28.388685 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740e08f-5c55-49d8-ae95-49ea42336656 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:28.390158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740e08f-5c55-49d8-ae95-49ea42336656 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:28.396731 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 136/275] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:31:28 2022] GET /identity/v3/role_assignments?user.id=0f5a82109d5348d2b94971c9bdb00c6d&scope.project.id=64dad222ddb64408b80bacb84185378f&role.id=fd2599a3f408404599ecea94959efc4a => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:28.397561 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740e08f-5c55-49d8-ae95-49ea42336656 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:28.400230 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:28.428275 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:28.508801 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbeb76-2612-4c25-a453-8772852e78e9 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:28.514368 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740e08f-5c55-49d8-ae95-49ea42336656 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:28.548666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c649c5-8d0a-4d0c-b219-f4ae437fda14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:28.549405 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c649c5-8d0a-4d0c-b219-f4ae437fda14 None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:28.549683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c649c5-8d0a-4d0c-b219-f4ae437fda14 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:28.549952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c649c5-8d0a-4d0c-b219-f4ae437fda14 None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:28.550662 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c649c5-8d0a-4d0c-b219-f4ae437fda14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:28.552468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c649c5-8d0a-4d0c-b219-f4ae437fda14 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:28.554197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106ea39-9500-4a13-a041-09d27b035595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:28.554925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106ea39-9500-4a13-a041-09d27b035595 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:28.555304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106ea39-9500-4a13-a041-09d27b035595 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:28.555575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106ea39-9500-4a13-a041-09d27b035595 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:28.556284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106ea39-9500-4a13-a041-09d27b035595 None admin] RBAC: Adding query filter params (name=member) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:28.556582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106ea39-9500-4a13-a041-09d27b035595 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:28.558670 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106ea39-9500-4a13-a041-09d27b035595 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:28.569103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 137/276] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:28.570465 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce974ad-b97f-442d-81be-92c140a0d490 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:28.574323 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce974ad-b97f-442d-81be-92c140a0d490 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:28.574323 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce974ad-b97f-442d-81be-92c140a0d490 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:28.574323 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 138/277] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:28.576032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce974ad-b97f-442d-81be-92c140a0d490 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:28.653699 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:28.669834 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 140/278] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Aug 8 16:31:28 2022] POST /identity/v3/users => generated 293 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:31:28.672415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152d8e8-d46e-46e8-8ddb-b3444db62e8f None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:28.672856 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152d8e8-d46e-46e8-8ddb-b3444db62e8f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:28.673273 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152d8e8-d46e-46e8-8ddb-b3444db62e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:28.700011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-3152d8e8-d46e-46e8-8ddb-b3444db62e8f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:28.701471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-3152d8e8-d46e-46e8-8ddb-b3444db62e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:28.725405 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce974ad-b97f-442d-81be-92c140a0d490 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:28.786959 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4862db5-c44b-401e-85cc-5502ffdae7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:28.788252 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4862db5-c44b-401e-85cc-5502ffdae7aa None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:28.788252 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4862db5-c44b-401e-85cc-5502ffdae7aa None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:28.788674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4862db5-c44b-401e-85cc-5502ffdae7aa None admin] PATH_INFO: `/v3/users/57b059fe91fe439abc9d17bf6be5c87a` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:28.802197 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 141/279] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:28.808348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4862db5-c44b-401e-85cc-5502ffdae7aa None admin] RBAC: Authorizing `identity:get_user(user_id=57b059fe91fe439abc9d17bf6be5c87a)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:28.809928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4862db5-c44b-401e-85cc-5502ffdae7aa None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:28.814614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 139/280] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:28 2022] GET /identity/v3/users/57b059fe91fe439abc9d17bf6be5c87a => generated 293 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:28.822741 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3152d8e8-d46e-46e8-8ddb-b3444db62e8f None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:28.827425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4862db5-c44b-401e-85cc-5502ffdae7aa None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:28.861650 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:28.945675 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3152d8e8-d46e-46e8-8ddb-b3444db62e8f None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:28.967650 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4862db5-c44b-401e-85cc-5502ffdae7aa None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:28.988492 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955c1e4-f777-4e93-ad5e-dc04fcb0ed1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:28.989113 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955c1e4-f777-4e93-ad5e-dc04fcb0ed1c None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:28.989392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955c1e4-f777-4e93-ad5e-dc04fcb0ed1c None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:28.989664 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955c1e4-f777-4e93-ad5e-dc04fcb0ed1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:28.990394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955c1e4-f777-4e93-ad5e-dc04fcb0ed1c None admin] RBAC: Adding query filter params (role.id=04bf40fcc26347de8167fc215f62c7aa, user.id=57b059fe91fe439abc9d17bf6be5c87a, scope.system=all) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:28.990759 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955c1e4-f777-4e93-ad5e-dc04fcb0ed1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:28.992658 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955c1e4-f777-4e93-ad5e-dc04fcb0ed1c None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:28.996361 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:29.000218 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 142/281] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Aug 8 16:31:28 2022] GET /identity/v3/role_assignments?user.id=57b059fe91fe439abc9d17bf6be5c87a&scope.system=all&role.id=04bf40fcc26347de8167fc215f62c7aa => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:29.013721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9f282-232d-4ddd-b3dd-c303c7ecc79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:29.014472 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9f282-232d-4ddd-b3dd-c303c7ecc79c None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:29.014848 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9f282-232d-4ddd-b3dd-c303c7ecc79c None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:29.015188 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9f282-232d-4ddd-b3dd-c303c7ecc79c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:29.035434 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9f282-232d-4ddd-b3dd-c303c7ecc79c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:29.040347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-d1b9f282-232d-4ddd-b3dd-c303c7ecc79c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:31:29.040347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:29.040347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:29.040347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:29.040347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:29.040347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:29.040347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:29.040347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:29.040347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:29.040347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:29.040347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:29.040347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:29.040347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:29.040347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:29.040347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:29.040347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:29.041420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:29.041420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:29.041420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:29.041420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:29.041420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:29.041420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:29.041420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:29.041420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:29.041420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:29.041420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:29.041420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:29.041420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:29.041420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:29.041420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:29.041420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:29.041420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:29.042422 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:29.042422 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:29.042422 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:29.042422 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:29.042422 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:29.042422 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:29.042422 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:29.042422 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:29.042422 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:29.042422 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:29.042422 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:31:29.042422 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:29.042422 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 140/282] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:31:29.049201 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955c1e4-f777-4e93-ad5e-dc04fcb0ed1c None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:29.173028 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:29.175975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955c1e4-f777-4e93-ad5e-dc04fcb0ed1c None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:29.214360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81daefbe-9a74-4bfd-934f-e3945a71c6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:29.215048 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81daefbe-9a74-4bfd-934f-e3945a71c6bc None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:29.215322 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81daefbe-9a74-4bfd-934f-e3945a71c6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:29.215587 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81daefbe-9a74-4bfd-934f-e3945a71c6bc None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:29.216251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81daefbe-9a74-4bfd-934f-e3945a71c6bc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:29.216544 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81daefbe-9a74-4bfd-934f-e3945a71c6bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:29.218160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81daefbe-9a74-4bfd-934f-e3945a71c6bc None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:29.227944 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 143/283] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:29.235670 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9f282-232d-4ddd-b3dd-c303c7ecc79c None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:29.355519 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9f282-232d-4ddd-b3dd-c303c7ecc79c None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:29.399862 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f27056-8469-4e73-b0dc-c4f40948c191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:29.401174 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f27056-8469-4e73-b0dc-c4f40948c191 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:29.401586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f27056-8469-4e73-b0dc-c4f40948c191 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:29.401869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f27056-8469-4e73-b0dc-c4f40948c191 None admin] PATH_INFO: `/v3/users/9f4b86e69f5b4c818485a62b75cb1fac` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:29.403054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f27056-8469-4e73-b0dc-c4f40948c191 None admin] RBAC: Authorizing `identity:get_user(user_id=9f4b86e69f5b4c818485a62b75cb1fac)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:29.404512 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f27056-8469-4e73-b0dc-c4f40948c191 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:29.406305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 141/284] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:29 2022] GET /identity/v3/users/9f4b86e69f5b4c818485a62b75cb1fac => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:29.412382 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81daefbe-9a74-4bfd-934f-e3945a71c6bc None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:29.570828 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81daefbe-9a74-4bfd-934f-e3945a71c6bc None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:29.622505 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f3bcc-93db-4828-9e8a-56cae99cfce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:29.623211 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f3bcc-93db-4828-9e8a-56cae99cfce4 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:29.623490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f3bcc-93db-4828-9e8a-56cae99cfce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:29.623851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f3bcc-93db-4828-9e8a-56cae99cfce4 None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:29.624926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f3bcc-93db-4828-9e8a-56cae99cfce4 None admin] RBAC: Authorizing `identity:get_project(project_id=64dad222ddb64408b80bacb84185378f)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:29.626374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f3bcc-93db-4828-9e8a-56cae99cfce4 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:29.628356 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 144/285] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:29 2022] GET /identity/v3/projects/64dad222ddb64408b80bacb84185378f => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:29.636855 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f27056-8469-4e73-b0dc-c4f40948c191 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:29.741788 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:29.768258 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d37303-7fdd-4fd5-a080-3503c43cab77 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:29.768563 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d37303-7fdd-4fd5-a080-3503c43cab77 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:29.768814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d37303-7fdd-4fd5-a080-3503c43cab77 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:29.770162 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 145/286] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:29.781621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd0324-5d85-49d8-8dbf-7a51fc45dfa8 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:29.781906 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd0324-5d85-49d8-8dbf-7a51fc45dfa8 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:29.782184 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd0324-5d85-49d8-8dbf-7a51fc45dfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:29.805030 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f27056-8469-4e73-b0dc-c4f40948c191 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:29.819623 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-9bbd0324-5d85-49d8-8dbf-7a51fc45dfa8 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:29.820907 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-9bbd0324-5d85-49d8-8dbf-7a51fc45dfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:29.843312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497c235-513e-4845-a1ee-114ef1976b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:29.844274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497c235-513e-4845-a1ee-114ef1976b9f None admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:29.844729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497c235-513e-4845-a1ee-114ef1976b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:29.845205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497c235-513e-4845-a1ee-114ef1976b9f None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f/users/9f4b86e69f5b4c818485a62b75cb1fac/roles/94f7f84c31e0425495dcca9783e7a46e` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:29.847499 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8497c235-513e-4845-a1ee-114ef1976b9f None admin] RBAC: Authorizing `identity:create_grant(project_id=64dad222ddb64408b80bacb84185378f, user_id=9f4b86e69f5b4c818485a62b75cb1fac, role_id=94f7f84c31e0425495dcca9783e7a46e)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:29.849152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8497c235-513e-4845-a1ee-114ef1976b9f None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:29.868814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:29.917927 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 146/287] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:29.937218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 142/288] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:31:29 2022] PUT /identity/v3/projects/64dad222ddb64408b80bacb84185378f/users/9f4b86e69f5b4c818485a62b75cb1fac/roles/94f7f84c31e0425495dcca9783e7a46e => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:31:29.950051 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd0324-5d85-49d8-8dbf-7a51fc45dfa8 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:30.065627 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd0324-5d85-49d8-8dbf-7a51fc45dfa8 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:30.105443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fdcdb9-d03c-4cad-b7f0-df603c955fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:30.106199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fdcdb9-d03c-4cad-b7f0-df603c955fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:30.106474 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fdcdb9-d03c-4cad-b7f0-df603c955fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:30.106844 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fdcdb9-d03c-4cad-b7f0-df603c955fa3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:30.118588 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fdcdb9-d03c-4cad-b7f0-df603c955fa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:30.123726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-e1fdcdb9-d03c-4cad-b7f0-df603c955fa3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:30.123726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:30.123726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:30.123726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:30.123726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:30.123726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:30.123726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:30.123726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:30.123726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:30.123726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:30.123726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:30.123726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:30.123726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:30.123726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:30.123726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:30.123726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:30.124696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:30.124696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:30.124696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:30.124696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:30.124696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:30.124696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:30.124696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:30.124696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:30.124696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:30.124696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:30.124696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:30.124696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:30.124696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:30.124696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:30.124696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:30.124696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:30.125573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:30.125573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:30.125573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:30.125573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:30.125573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:30.125573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:30.125573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:30.125573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:30.125573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:30.125573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:30.125573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:30.125573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:30.125573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 147/289] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:30.131260 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497c235-513e-4845-a1ee-114ef1976b9f None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:30.244934 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497c235-513e-4845-a1ee-114ef1976b9f None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:30.256529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:30.283155 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58dd50-1905-4680-92b0-a1e8568bd790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:30.283946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58dd50-1905-4680-92b0-a1e8568bd790 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:30.284226 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58dd50-1905-4680-92b0-a1e8568bd790 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:30.284497 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58dd50-1905-4680-92b0-a1e8568bd790 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:30.285255 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de58dd50-1905-4680-92b0-a1e8568bd790 None admin] RBAC: Adding query filter params (name=member) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:30.285558 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de58dd50-1905-4680-92b0-a1e8568bd790 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:30.286991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de58dd50-1905-4680-92b0-a1e8568bd790 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:30.296302 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 143/290] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:30.303069 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fdcdb9-d03c-4cad-b7f0-df603c955fa3 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:30.329530 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:30.422560 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fdcdb9-d03c-4cad-b7f0-df603c955fa3 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:30.462728 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bc7cd-1811-4c30-9adb-68f60af3be76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:30.464290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bc7cd-1811-4c30-9adb-68f60af3be76 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:30.464575 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bc7cd-1811-4c30-9adb-68f60af3be76 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:30.464847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bc7cd-1811-4c30-9adb-68f60af3be76 None admin] PATH_INFO: `/v3/users/fcdda2eb8bce465aab67d0d8a16d2615` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:30.466106 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bc7cd-1811-4c30-9adb-68f60af3be76 None admin] RBAC: Authorizing `identity:get_user(user_id=fcdda2eb8bce465aab67d0d8a16d2615)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:30.467925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bc7cd-1811-4c30-9adb-68f60af3be76 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:30.469858 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 148/291] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:30 2022] GET /identity/v3/users/fcdda2eb8bce465aab67d0d8a16d2615 => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:30.475630 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58dd50-1905-4680-92b0-a1e8568bd790 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:30.584892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58dd50-1905-4680-92b0-a1e8568bd790 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:30.625295 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ec436-2f34-4199-860e-87b3ae28c90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:30.625904 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ec436-2f34-4199-860e-87b3ae28c90d None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:30.626176 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ec436-2f34-4199-860e-87b3ae28c90d None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:30.626665 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ec436-2f34-4199-860e-87b3ae28c90d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:30.674635 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:30.702067 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ec436-2f34-4199-860e-87b3ae28c90d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:30.703604 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ec436-2f34-4199-860e-87b3ae28c90d None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:30.813890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 144/292] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 16:31:30.819791 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bc7cd-1811-4c30-9adb-68f60af3be76 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:30.915751 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:30.935228 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bc7cd-1811-4c30-9adb-68f60af3be76 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:30.969984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2efbfc9-ba6d-4f42-be60-9402cfc32fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:30.970639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efbfc9-ba6d-4f42-be60-9402cfc32fbe None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:30.970916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efbfc9-ba6d-4f42-be60-9402cfc32fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:30.971183 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efbfc9-ba6d-4f42-be60-9402cfc32fbe None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:30.972299 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efbfc9-ba6d-4f42-be60-9402cfc32fbe None admin] RBAC: Authorizing `identity:get_project(project_id=64dad222ddb64408b80bacb84185378f)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:30.973727 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efbfc9-ba6d-4f42-be60-9402cfc32fbe None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:30.975864 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 149/293] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:30 2022] GET /identity/v3/projects/64dad222ddb64408b80bacb84185378f => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:30.982865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ec436-2f34-4199-860e-87b3ae28c90d None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:31.059439 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:31.091526 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ec436-2f34-4199-860e-87b3ae28c90d None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:31.129104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3005a-48ea-44b3-94bd-7e1543d3f76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:31.129705 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3005a-48ea-44b3-94bd-7e1543d3f76a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:31.129978 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3005a-48ea-44b3-94bd-7e1543d3f76a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:31.130242 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3005a-48ea-44b3-94bd-7e1543d3f76a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:31.131045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3005a-48ea-44b3-94bd-7e1543d3f76a None admin] RBAC: Adding query filter params (role.id=04bf40fcc26347de8167fc215f62c7aa, scope.project.id=64dad222ddb64408b80bacb84185378f, user.id=fcdda2eb8bce465aab67d0d8a16d2615) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:31.131345 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3005a-48ea-44b3-94bd-7e1543d3f76a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:31.132792 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3005a-48ea-44b3-94bd-7e1543d3f76a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:31.140259 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 145/294] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:31:30 2022] GET /identity/v3/role_assignments?user.id=fcdda2eb8bce465aab67d0d8a16d2615&scope.project.id=64dad222ddb64408b80bacb84185378f&role.id=04bf40fcc26347de8167fc215f62c7aa => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:31.455888 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d320fea-68d8-42be-8538-cc5df190c199 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:31.456397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d320fea-68d8-42be-8538-cc5df190c199 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:31.456397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d320fea-68d8-42be-8538-cc5df190c199 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:31.457997 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 150/295] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:31.463886 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c192459-3db6-425e-bb13-8f5d9f53ecee None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:31.464240 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c192459-3db6-425e-bb13-8f5d9f53ecee None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:31.464715 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c192459-3db6-425e-bb13-8f5d9f53ecee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:31.490518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-6c192459-3db6-425e-bb13-8f5d9f53ecee None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:31.492616 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-6c192459-3db6-425e-bb13-8f5d9f53ecee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:31.533314 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1814c78-a484-4517-a93f-2706a2a22ba9 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:31.533611 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1814c78-a484-4517-a93f-2706a2a22ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:31.533834 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1814c78-a484-4517-a93f-2706a2a22ba9 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:31.535474 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 151/296] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:31.540444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd794cf-6185-4a07-b08c-6da54b1ff783 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:31.540739 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd794cf-6185-4a07-b08c-6da54b1ff783 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:31.540999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd794cf-6185-4a07-b08c-6da54b1ff783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:31.569866 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-3dd794cf-6185-4a07-b08c-6da54b1ff783 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:31.571224 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-3dd794cf-6185-4a07-b08c-6da54b1ff783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:31.586288 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 146/297] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:31.615291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c192459-3db6-425e-bb13-8f5d9f53ecee None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:31.669754 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 152/298] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:31.691250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:31.694955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd794cf-6185-4a07-b08c-6da54b1ff783 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:31.742968 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c192459-3db6-425e-bb13-8f5d9f53ecee None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:31.785481 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a569c-fe86-4754-a4bc-04ccc4d49ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:31.786117 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a569c-fe86-4754-a4bc-04ccc4d49ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:31.786401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a569c-fe86-4754-a4bc-04ccc4d49ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:31.786693 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a569c-fe86-4754-a4bc-04ccc4d49ba5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:31.792672 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:31.797546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a569c-fe86-4754-a4bc-04ccc4d49ba5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:31.802273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-847a569c-fe86-4754-a4bc-04ccc4d49ba5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:31:31.802273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:31.802273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:31.802273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:31.802273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:31.802273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:31.802273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:31.802273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:31.802273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:31.802273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:31.802273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:31.802273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:31.802273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:31.802273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:31.802273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:31.802273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:31.811157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:31.811157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:31.811157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:31.811157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:31.811157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:31.811157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:31.811157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:31.811157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:31.811157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:31.811157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:31.811157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:31.811157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:31.811157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:31.811157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:31.811157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:31.811157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:31.811993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:31.811993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:31.811993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:31.811993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:31.811993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:31.811993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:31.811993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:31.811993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:31.811993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:31.811993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:31.811993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:31:31.811993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:31.811993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 147/299] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:31:31.811993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bc051-11f9-443f-9f6d-e92c9bf9526b None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:31.812715 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bc051-11f9-443f-9f6d-e92c9bf9526b None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:31.812715 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bc051-11f9-443f-9f6d-e92c9bf9526b None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:31.812715 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 148/300] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:31.814226 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3bc051-11f9-443f-9f6d-e92c9bf9526b None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:31.816966 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd794cf-6185-4a07-b08c-6da54b1ff783 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:31.859938 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f1a534-f1d1-4472-9035-ddc1df485b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:31.861033 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1a534-f1d1-4472-9035-ddc1df485b3e None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:31.861313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1a534-f1d1-4472-9035-ddc1df485b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:31.861581 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1a534-f1d1-4472-9035-ddc1df485b3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:31.863396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f1a534-f1d1-4472-9035-ddc1df485b3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:31.865100 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f1a534-f1d1-4472-9035-ddc1df485b3e None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:31.870905 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 153/301] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:31:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:31.872989 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278da29-8f44-40c3-af46-8c7ef22417d1 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:31.873269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278da29-8f44-40c3-af46-8c7ef22417d1 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:31.873694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278da29-8f44-40c3-af46-8c7ef22417d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:31.892206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-d278da29-8f44-40c3-af46-8c7ef22417d1 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:31.893486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-d278da29-8f44-40c3-af46-8c7ef22417d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:31.929323 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3bc051-11f9-443f-9f6d-e92c9bf9526b None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:31.957109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:31.966764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0a4ad-e776-4a2c-ab6e-a8c9b847bdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:31.967359 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0a4ad-e776-4a2c-ab6e-a8c9b847bdce None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:31.967638 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0a4ad-e776-4a2c-ab6e-a8c9b847bdce None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:31.968171 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0a4ad-e776-4a2c-ab6e-a8c9b847bdce None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:31.971640 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0a4ad-e776-4a2c-ab6e-a8c9b847bdce None admin] RBAC: Adding query filter params (name=reader) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:31.971640 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0a4ad-e776-4a2c-ab6e-a8c9b847bdce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:31.971640 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0a4ad-e776-4a2c-ab6e-a8c9b847bdce None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:31.979622 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 154/302] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:31.979622 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278da29-8f44-40c3-af46-8c7ef22417d1 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:31.982925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 149/303] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:31.988804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0a4ad-e776-4a2c-ab6e-a8c9b847bdce None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:32.073434 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:32.090942 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278da29-8f44-40c3-af46-8c7ef22417d1 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:32.095238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0a4ad-e776-4a2c-ab6e-a8c9b847bdce None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:32.129790 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f40af-e242-4d2a-a723-9af7c1f6d8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:32.130405 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f40af-e242-4d2a-a723-9af7c1f6d8c4 None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:32.130725 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f40af-e242-4d2a-a723-9af7c1f6d8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:32.130922 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db0157-0d6c-439c-b1a5-30af3dff3fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:32.131032 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f40af-e242-4d2a-a723-9af7c1f6d8c4 None admin] PATH_INFO: `/v3/groups` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:32.131549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db0157-0d6c-439c-b1a5-30af3dff3fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:32.131657 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f40af-e242-4d2a-a723-9af7c1f6d8c4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:32.131829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db0157-0d6c-439c-b1a5-30af3dff3fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:32.132099 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db0157-0d6c-439c-b1a5-30af3dff3fe0 None admin] PATH_INFO: `/v3/users/8c47821d07804dd78baadaa7a12b9c48` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:32.133102 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f40af-e242-4d2a-a723-9af7c1f6d8c4 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:32.146782 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db0157-0d6c-439c-b1a5-30af3dff3fe0 None admin] RBAC: Authorizing `identity:get_user(user_id=8c47821d07804dd78baadaa7a12b9c48)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:32.148251 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db0157-0d6c-439c-b1a5-30af3dff3fe0 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:32.149909 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 150/304] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:31 2022] GET /identity/v3/users/8c47821d07804dd78baadaa7a12b9c48 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:32.151357 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db0157-0d6c-439c-b1a5-30af3dff3fe0 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:32.197141 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:32.208662 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 155/305] 10.222.0.13 () {60 vars in 1134 bytes} [Mon Aug 8 16:31:31 2022] POST /identity/v3/groups => generated 229 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:31:32.219638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f40af-e242-4d2a-a723-9af7c1f6d8c4 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:32.272097 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db0157-0d6c-439c-b1a5-30af3dff3fe0 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:32.318771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d5ae2-923f-4423-9df9-a1111f0733cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:32.319773 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d5ae2-923f-4423-9df9-a1111f0733cc None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:32.320244 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d5ae2-923f-4423-9df9-a1111f0733cc None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:32.320749 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d5ae2-923f-4423-9df9-a1111f0733cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:32.334111 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d5ae2-923f-4423-9df9-a1111f0733cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:32.340048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-ed1d5ae2-923f-4423-9df9-a1111f0733cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:32.340048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:32.340048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:32.340048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:32.340048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:32.340048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:32.340048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:32.340048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:32.340048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:32.340048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:32.340048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:32.340048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:32.340048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:32.340048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:32.340048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:32.340048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:32.341009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:32.341009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:32.341009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:32.341009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:32.341009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:32.341009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:32.341009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:32.341009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:32.341009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:32.341009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:32.341009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:32.341009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:32.341009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:32.341009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:32.341009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:32.341009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:32.341810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:32.341810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:32.341810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:32.341810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:32.341810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:32.341810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:32.341810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:32.341810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:32.341810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:32.341810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:32.341810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:32.341810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:32.344495 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 151/306] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:32.350527 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d5ae2-923f-4423-9df9-a1111f0733cc None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:32.359827 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f40af-e242-4d2a-a723-9af7c1f6d8c4 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:32.397821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41086302-7623-4318-8faf-90401ecf50ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:32.398436 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41086302-7623-4318-8faf-90401ecf50ae None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:32.399798 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41086302-7623-4318-8faf-90401ecf50ae None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:32.399933 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41086302-7623-4318-8faf-90401ecf50ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:32.400654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41086302-7623-4318-8faf-90401ecf50ae None admin] RBAC: Adding query filter params (role.id=94f7f84c31e0425495dcca9783e7a46e, user.id=8c47821d07804dd78baadaa7a12b9c48, scope.system=all) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:32.400950 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41086302-7623-4318-8faf-90401ecf50ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:32.402389 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41086302-7623-4318-8faf-90401ecf50ae None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:32.410508 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 156/307] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Aug 8 16:31:32 2022] GET /identity/v3/role_assignments?user.id=8c47821d07804dd78baadaa7a12b9c48&scope.system=all&role.id=94f7f84c31e0425495dcca9783e7a46e => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:32.477130 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d5ae2-923f-4423-9df9-a1111f0733cc None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:32.487230 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:32.493656 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993b3cb-9685-4676-bf1d-1b9fe2009fba None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:32.494870 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993b3cb-9685-4676-bf1d-1b9fe2009fba None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:32.494870 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993b3cb-9685-4676-bf1d-1b9fe2009fba None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:32.496447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 157/308] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:32.502260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bdc9a7-c4f5-44e8-ae1f-6651622d49bd None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:32.502552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bdc9a7-c4f5-44e8-ae1f-6651622d49bd None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:32.502829 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bdc9a7-c4f5-44e8-ae1f-6651622d49bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:32.524974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-50bdc9a7-c4f5-44e8-ae1f-6651622d49bd None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:32.527120 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-50bdc9a7-c4f5-44e8-ae1f-6651622d49bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:32.530430 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:32.537510 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a3a6c-9093-4fe4-97b3-80306b3f7817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:32.538775 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a3a6c-9093-4fe4-97b3-80306b3f7817 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:32.539056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a3a6c-9093-4fe4-97b3-80306b3f7817 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:32.539327 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a3a6c-9093-4fe4-97b3-80306b3f7817 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:32.542763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0a3a6c-9093-4fe4-97b3-80306b3f7817 None admin] RBAC: Adding query filter params (name=member) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:32.542763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0a3a6c-9093-4fe4-97b3-80306b3f7817 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:32.543512 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0a3a6c-9093-4fe4-97b3-80306b3f7817 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:32.556368 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 152/309] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:32.563369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a3a6c-9093-4fe4-97b3-80306b3f7817 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:32.622706 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 158/310] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:31:32.645258 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bdc9a7-c4f5-44e8-ae1f-6651622d49bd None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:32.680018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a3a6c-9093-4fe4-97b3-80306b3f7817 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:32.725858 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b283f0b-10a7-4c05-81e1-15016cee0034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:32.726501 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b283f0b-10a7-4c05-81e1-15016cee0034 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:32.726932 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b283f0b-10a7-4c05-81e1-15016cee0034 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:32.727200 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b283f0b-10a7-4c05-81e1-15016cee0034 None admin] PATH_INFO: `/v3/users/57b059fe91fe439abc9d17bf6be5c87a` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:32.728395 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b283f0b-10a7-4c05-81e1-15016cee0034 None admin] RBAC: Authorizing `identity:get_user(user_id=57b059fe91fe439abc9d17bf6be5c87a)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:32.730526 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b283f0b-10a7-4c05-81e1-15016cee0034 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:32.732174 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 153/311] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:32 2022] GET /identity/v3/users/57b059fe91fe439abc9d17bf6be5c87a => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:32.738000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b283f0b-10a7-4c05-81e1-15016cee0034 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:32.752881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bdc9a7-c4f5-44e8-ae1f-6651622d49bd None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:32.782812 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:32.790933 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82fb01-21f2-4b91-8aff-1d1af06d2d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:32.792140 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82fb01-21f2-4b91-8aff-1d1af06d2d19 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:32.795983 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82fb01-21f2-4b91-8aff-1d1af06d2d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:32.795983 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82fb01-21f2-4b91-8aff-1d1af06d2d19 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:32.803670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac82fb01-21f2-4b91-8aff-1d1af06d2d19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:32.808573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-ac82fb01-21f2-4b91-8aff-1d1af06d2d19 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:31:32.808573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:32.808573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:32.808573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:32.808573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:32.808573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:32.808573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:32.808573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:32.808573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:32.808573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:32.808573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:32.808573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:32.808573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:32.808573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:32.808573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:32.808573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:32.809504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:32.809504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:32.809504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:32.809504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:32.809504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:32.809504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:32.809504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:32.809504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:32.809504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:32.809504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:32.809504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:32.809504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:32.809504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:32.809504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:32.809504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:32.809504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:32.810384 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:32.810384 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:32.810384 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:32.810384 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:32.810384 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:32.810384 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:32.810384 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:32.810384 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:32.810384 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:32.810384 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:32.810384 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:31:32.810384 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:32.811059 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:32.814669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 159/312] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:31:32.816262 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82fb01-21f2-4b91-8aff-1d1af06d2d19 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:32.845754 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b283f0b-10a7-4c05-81e1-15016cee0034 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:32.881293 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6dfd2-78d1-4ebf-8f0f-502599bbbf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:32.881910 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6dfd2-78d1-4ebf-8f0f-502599bbbf7c None admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:32.882189 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6dfd2-78d1-4ebf-8f0f-502599bbbf7c None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:32.882456 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6dfd2-78d1-4ebf-8f0f-502599bbbf7c None admin] PATH_INFO: `/v3/system/users/57b059fe91fe439abc9d17bf6be5c87a/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:32.885523 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b6dfd2-78d1-4ebf-8f0f-502599bbbf7c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=57b059fe91fe439abc9d17bf6be5c87a, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:32.886940 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b6dfd2-78d1-4ebf-8f0f-502599bbbf7c None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:32.922141 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82fb01-21f2-4b91-8aff-1d1af06d2d19 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:32.958757 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49cb39-b247-409f-ae1f-b9bb003265c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:32.959370 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49cb39-b247-409f-ae1f-b9bb003265c8 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:32.959646 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49cb39-b247-409f-ae1f-b9bb003265c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:32.959941 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49cb39-b247-409f-ae1f-b9bb003265c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:32.960602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd49cb39-b247-409f-ae1f-b9bb003265c8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:32.960956 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd49cb39-b247-409f-ae1f-b9bb003265c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:32.962328 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd49cb39-b247-409f-ae1f-b9bb003265c8 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:32.972164 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 160/313] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:32.977512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49cb39-b247-409f-ae1f-b9bb003265c8 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:32.992004 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 154/314] 10.222.0.13 () {58 vars in 1334 bytes} [Mon Aug 8 16:31:32 2022] PUT /identity/v3/system/users/57b059fe91fe439abc9d17bf6be5c87a/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:31:33.051979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:33.103009 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49cb39-b247-409f-ae1f-b9bb003265c8 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:33.142268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e805e-c9e3-40c2-ad3c-df7b5894e218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:33.143334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e805e-c9e3-40c2-ad3c-df7b5894e218 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:33.143931 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e805e-c9e3-40c2-ad3c-df7b5894e218 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:33.144407 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e805e-c9e3-40c2-ad3c-df7b5894e218 None admin] PATH_INFO: `/v3/users/9f4b86e69f5b4c818485a62b75cb1fac` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:33.145839 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e805e-c9e3-40c2-ad3c-df7b5894e218 None admin] RBAC: Authorizing `identity:get_user(user_id=9f4b86e69f5b4c818485a62b75cb1fac)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:33.147573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e805e-c9e3-40c2-ad3c-df7b5894e218 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:33.149714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 161/315] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:32 2022] GET /identity/v3/users/9f4b86e69f5b4c818485a62b75cb1fac => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:33.155746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6dfd2-78d1-4ebf-8f0f-502599bbbf7c None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:33.266858 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:33.270297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6dfd2-78d1-4ebf-8f0f-502599bbbf7c None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:33.310686 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584111ab-862c-47d4-9716-c83326c16997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:33.311366 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584111ab-862c-47d4-9716-c83326c16997 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:33.311644 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584111ab-862c-47d4-9716-c83326c16997 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:33.311917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584111ab-862c-47d4-9716-c83326c16997 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:33.383521 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584111ab-862c-47d4-9716-c83326c16997 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:33.385603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584111ab-862c-47d4-9716-c83326c16997 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:33.495400 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:33.495797 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 155/316] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 16:31:33.502072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e805e-c9e3-40c2-ad3c-df7b5894e218 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:33.627870 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e805e-c9e3-40c2-ad3c-df7b5894e218 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:33.660896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:33.669512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5dca77-c434-4b0d-96bc-5156f3dfbd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:33.670155 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5dca77-c434-4b0d-96bc-5156f3dfbd10 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:33.670435 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5dca77-c434-4b0d-96bc-5156f3dfbd10 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:33.670739 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5dca77-c434-4b0d-96bc-5156f3dfbd10 None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:33.671897 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5dca77-c434-4b0d-96bc-5156f3dfbd10 None admin] RBAC: Authorizing `identity:get_project(project_id=64dad222ddb64408b80bacb84185378f)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:33.673357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5dca77-c434-4b0d-96bc-5156f3dfbd10 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:33.675324 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 162/317] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:33 2022] GET /identity/v3/projects/64dad222ddb64408b80bacb84185378f => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:33.683628 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584111ab-862c-47d4-9716-c83326c16997 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:33.788204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584111ab-862c-47d4-9716-c83326c16997 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:33.825081 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70629951-7878-479e-9be6-03e1f21b4ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:33.825740 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70629951-7878-479e-9be6-03e1f21b4ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:33.826023 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70629951-7878-479e-9be6-03e1f21b4ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:33.826295 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70629951-7878-479e-9be6-03e1f21b4ae8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:33.827051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70629951-7878-479e-9be6-03e1f21b4ae8 None admin] RBAC: Adding query filter params (role.id=94f7f84c31e0425495dcca9783e7a46e, scope.project.id=64dad222ddb64408b80bacb84185378f, user.id=9f4b86e69f5b4c818485a62b75cb1fac) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:33.827359 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70629951-7878-479e-9be6-03e1f21b4ae8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:33.828805 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70629951-7878-479e-9be6-03e1f21b4ae8 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:33.834861 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 156/318] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:31:33 2022] GET /identity/v3/role_assignments?user.id=9f4b86e69f5b4c818485a62b75cb1fac&scope.project.id=64dad222ddb64408b80bacb84185378f&role.id=94f7f84c31e0425495dcca9783e7a46e => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:34.416670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a40655-bb8b-4633-bfa9-c3ce9038031f None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:34.417105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a40655-bb8b-4633-bfa9-c3ce9038031f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:34.417280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a40655-bb8b-4633-bfa9-c3ce9038031f None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:34.418626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 163/319] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:34.425984 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581d13d-cc16-4b55-bd62-9458c7eba54d None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:34.425984 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581d13d-cc16-4b55-bd62-9458c7eba54d None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:34.425984 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581d13d-cc16-4b55-bd62-9458c7eba54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:34.443414 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-4581d13d-cc16-4b55-bd62-9458c7eba54d None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:34.444925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-4581d13d-cc16-4b55-bd62-9458c7eba54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:34.466698 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5abcac-d1d2-4e2d-9b93-76378a6e3e22 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:34.466982 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5abcac-d1d2-4e2d-9b93-76378a6e3e22 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:34.467247 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5abcac-d1d2-4e2d-9b93-76378a6e3e22 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:34.469009 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 164/320] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:31:34.473868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d15d58-b7ff-4d39-bcb2-735fa71daafa None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:34.474152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d15d58-b7ff-4d39-bcb2-735fa71daafa None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:34.474426 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d15d58-b7ff-4d39-bcb2-735fa71daafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:34.496592 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-31d15d58-b7ff-4d39-bcb2-735fa71daafa None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:34.498735 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-31d15d58-b7ff-4d39-bcb2-735fa71daafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:34.516735 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:34.525076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 157/321] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:34.549266 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581d13d-cc16-4b55-bd62-9458c7eba54d None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:34.575202 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 165/322] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:31:34.599524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d15d58-b7ff-4d39-bcb2-735fa71daafa None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:34.659600 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581d13d-cc16-4b55-bd62-9458c7eba54d None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:34.696811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee159bb3-6c08-4996-82e3-532e86516232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:34.698434 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee159bb3-6c08-4996-82e3-532e86516232 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:34.698434 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee159bb3-6c08-4996-82e3-532e86516232 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:34.698434 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee159bb3-6c08-4996-82e3-532e86516232 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:34.699594 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee159bb3-6c08-4996-82e3-532e86516232 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:34.701051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee159bb3-6c08-4996-82e3-532e86516232 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:34.702783 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 158/323] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:31:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:34.706755 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d15d58-b7ff-4d39-bcb2-735fa71daafa None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:34.708496 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee159bb3-6c08-4996-82e3-532e86516232 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:34.737033 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:34.747242 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760e8eb-c67d-456f-8616-cba82b4a0172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:34.747907 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760e8eb-c67d-456f-8616-cba82b4a0172 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:34.748188 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760e8eb-c67d-456f-8616-cba82b4a0172 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:34.748456 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760e8eb-c67d-456f-8616-cba82b4a0172 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:34.761453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760e8eb-c67d-456f-8616-cba82b4a0172 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:34.766936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-9760e8eb-c67d-456f-8616-cba82b4a0172 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:31:34.766936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:34.766936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:34.766936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:34.766936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:34.766936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:34.766936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:34.766936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:34.766936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:34.766936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:34.766936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:34.766936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:34.766936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:34.766936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:34.766936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:34.766936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:34.767964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:34.767964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:34.767964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:34.767964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:34.767964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:34.767964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:34.767964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:34.767964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:34.767964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:34.767964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:34.767964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:34.767964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:34.767964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:34.767964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:34.767964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:34.767964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:34.768773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:34.768773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:34.768773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:34.768773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:34.768773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:34.768773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:34.768773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:34.768773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:34.768773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:34.768773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:34.768773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:31:34.768773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:34.771597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 166/324] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:31:34.777919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760e8eb-c67d-456f-8616-cba82b4a0172 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:34.792785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:34.835657 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee159bb3-6c08-4996-82e3-532e86516232 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:34.876626 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16de83-d3b3-45cf-b9e7-685fca22724e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:34.877245 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16de83-d3b3-45cf-b9e7-685fca22724e None admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:34.877519 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16de83-d3b3-45cf-b9e7-685fca22724e None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:34.877780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16de83-d3b3-45cf-b9e7-685fca22724e None admin] PATH_INFO: `/v3/groups` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:34.878412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16de83-d3b3-45cf-b9e7-685fca22724e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:34.879996 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16de83-d3b3-45cf-b9e7-685fca22724e None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:34.899928 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760e8eb-c67d-456f-8616-cba82b4a0172 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:34.939129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbda9ac-b63f-4df2-a91a-bee8274d8303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:34.939752 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbda9ac-b63f-4df2-a91a-bee8274d8303 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:34.940021 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbda9ac-b63f-4df2-a91a-bee8274d8303 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:34.940285 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbda9ac-b63f-4df2-a91a-bee8274d8303 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:34.941038 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbda9ac-b63f-4df2-a91a-bee8274d8303 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:34.941333 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbda9ac-b63f-4df2-a91a-bee8274d8303 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:34.942746 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbda9ac-b63f-4df2-a91a-bee8274d8303 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:34.953215 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 167/325] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:34.959013 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbda9ac-b63f-4df2-a91a-bee8274d8303 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:34.978060 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 159/326] 10.222.0.13 () {60 vars in 1134 bytes} [Mon Aug 8 16:31:34 2022] POST /identity/v3/groups => generated 226 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:31:35.021618 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:35.086053 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbda9ac-b63f-4df2-a91a-bee8274d8303 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:35.127849 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254302e-3821-4cbd-b586-23437767d794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:35.128506 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254302e-3821-4cbd-b586-23437767d794 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:35.128782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254302e-3821-4cbd-b586-23437767d794 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:35.129053 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254302e-3821-4cbd-b586-23437767d794 None admin] PATH_INFO: `/v3/users/8c47821d07804dd78baadaa7a12b9c48` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:35.130244 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254302e-3821-4cbd-b586-23437767d794 None admin] RBAC: Authorizing `identity:get_user(user_id=8c47821d07804dd78baadaa7a12b9c48)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:35.131752 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254302e-3821-4cbd-b586-23437767d794 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:35.133615 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 168/327] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:34 2022] GET /identity/v3/users/8c47821d07804dd78baadaa7a12b9c48 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:35.139837 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16de83-d3b3-45cf-b9e7-685fca22724e None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:35.244796 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d158999-fb81-49e4-a683-d53035e8637c None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:35.245089 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d158999-fb81-49e4-a683-d53035e8637c None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:35.246855 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d158999-fb81-49e4-a683-d53035e8637c None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:35.247035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 169/328] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:31:35.252202 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f991f04-c275-4ef9-bb75-c3a46090ba2d None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:35.252469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f991f04-c275-4ef9-bb75-c3a46090ba2d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:35.252735 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f991f04-c275-4ef9-bb75-c3a46090ba2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:35.264419 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16de83-d3b3-45cf-b9e7-685fca22724e None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:35.273667 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-2f991f04-c275-4ef9-bb75-c3a46090ba2d None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:35.274901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-2f991f04-c275-4ef9-bb75-c3a46090ba2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:35.297107 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:35.309879 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8823de-c255-49ad-8b67-6e34d5c3a914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:35.309879 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8823de-c255-49ad-8b67-6e34d5c3a914 None admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:35.310137 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8823de-c255-49ad-8b67-6e34d5c3a914 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:35.313183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8823de-c255-49ad-8b67-6e34d5c3a914 None admin] PATH_INFO: `/v3/system/users/8c47821d07804dd78baadaa7a12b9c48/roles/94f7f84c31e0425495dcca9783e7a46e` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:35.313183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8823de-c255-49ad-8b67-6e34d5c3a914 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8c47821d07804dd78baadaa7a12b9c48, role_id=94f7f84c31e0425495dcca9783e7a46e)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:35.313183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8823de-c255-49ad-8b67-6e34d5c3a914 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:35.364209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 170/329] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:31:35.384065 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 160/330] 10.222.0.13 () {58 vars in 1334 bytes} [Mon Aug 8 16:31:35 2022] PUT /identity/v3/system/users/8c47821d07804dd78baadaa7a12b9c48/roles/94f7f84c31e0425495dcca9783e7a46e => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:31:35.388787 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f991f04-c275-4ef9-bb75-c3a46090ba2d None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:35.447745 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:35.519963 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f991f04-c275-4ef9-bb75-c3a46090ba2d None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:35.566020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ebf9a0-dedf-4caf-8cdf-e38c63adee5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:35.566722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebf9a0-dedf-4caf-8cdf-e38c63adee5a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:35.567009 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebf9a0-dedf-4caf-8cdf-e38c63adee5a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:35.567284 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebf9a0-dedf-4caf-8cdf-e38c63adee5a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:35.581820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ebf9a0-dedf-4caf-8cdf-e38c63adee5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:35.586868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-64ebf9a0-dedf-4caf-8cdf-e38c63adee5a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:35.586868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:35.586868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:35.586868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:35.586868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:35.586868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:35.586868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:35.586868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:35.586868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:35.586868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:35.586868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:35.586868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:35.586868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:35.586868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:35.586868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:35.586868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:35.587812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:35.587812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:35.587812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:35.587812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:35.587812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:35.587812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:35.587812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:35.587812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:35.587812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:35.587812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:35.587812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:35.587812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:35.587812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:35.587812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:35.587812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:35.587812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:35.589551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:35.589551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:35.589551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:35.589551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:35.589551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:35.589551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:35.589551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:35.589551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:35.589551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:35.589551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:35.589551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:35.589551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:35.589551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 171/331] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:35.595534 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8823de-c255-49ad-8b67-6e34d5c3a914 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:35.722972 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8823de-c255-49ad-8b67-6e34d5c3a914 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:35.775838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc092710-36ec-49ba-8da5-65a0b68f4cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:35.776457 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc092710-36ec-49ba-8da5-65a0b68f4cbc None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:35.776773 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc092710-36ec-49ba-8da5-65a0b68f4cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:35.776993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc092710-36ec-49ba-8da5-65a0b68f4cbc None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:35.777669 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc092710-36ec-49ba-8da5-65a0b68f4cbc None admin] RBAC: Adding query filter params (name=member) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:35.777966 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc092710-36ec-49ba-8da5-65a0b68f4cbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:35.780179 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc092710-36ec-49ba-8da5-65a0b68f4cbc None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:35.789832 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 161/332] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:35.797448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ebf9a0-dedf-4caf-8cdf-e38c63adee5a None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:35.923602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ebf9a0-dedf-4caf-8cdf-e38c63adee5a None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:35.934921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:35.969969 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e49a2-10ac-4192-b676-18cbb01d5783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:35.970645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e49a2-10ac-4192-b676-18cbb01d5783 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:35.970924 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e49a2-10ac-4192-b676-18cbb01d5783 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:35.971192 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e49a2-10ac-4192-b676-18cbb01d5783 None admin] PATH_INFO: `/v3/users/57b059fe91fe439abc9d17bf6be5c87a` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:35.975282 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834e49a2-10ac-4192-b676-18cbb01d5783 None admin] RBAC: Authorizing `identity:get_user(user_id=57b059fe91fe439abc9d17bf6be5c87a)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:35.976756 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834e49a2-10ac-4192-b676-18cbb01d5783 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:35.979349 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 172/333] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:35 2022] GET /identity/v3/users/57b059fe91fe439abc9d17bf6be5c87a => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:35.991706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc092710-36ec-49ba-8da5-65a0b68f4cbc None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:36.075057 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:36.128805 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc092710-36ec-49ba-8da5-65a0b68f4cbc None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:36.176086 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ec206-3572-49a0-ac68-32bd7d02a720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:36.176691 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ec206-3572-49a0-ac68-32bd7d02a720 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:36.176959 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ec206-3572-49a0-ac68-32bd7d02a720 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:36.177221 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ec206-3572-49a0-ac68-32bd7d02a720 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:36.177940 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ec206-3572-49a0-ac68-32bd7d02a720 None admin] RBAC: Adding query filter params (role.id=04bf40fcc26347de8167fc215f62c7aa, user.id=57b059fe91fe439abc9d17bf6be5c87a, scope.system=all) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:36.178244 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ec206-3572-49a0-ac68-32bd7d02a720 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:36.179703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ec206-3572-49a0-ac68-32bd7d02a720 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:36.197306 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 162/334] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Aug 8 16:31:35 2022] GET /identity/v3/role_assignments?user.id=57b059fe91fe439abc9d17bf6be5c87a&scope.system=all&role.id=04bf40fcc26347de8167fc215f62c7aa => generated 510 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:37.892618 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5eee7-be26-48c6-b667-f00bed9897a5 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:37.893199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5eee7-be26-48c6-b667-f00bed9897a5 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:37.893199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5eee7-be26-48c6-b667-f00bed9897a5 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:37.894782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 173/335] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:37.901116 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abef248-dd45-4e39-a13b-0406b5777fe0 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:37.901636 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abef248-dd45-4e39-a13b-0406b5777fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:37.901636 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abef248-dd45-4e39-a13b-0406b5777fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:37.932240 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-1abef248-dd45-4e39-a13b-0406b5777fe0 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:37.933633 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-1abef248-dd45-4e39-a13b-0406b5777fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:38.021189 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 163/336] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:38.070864 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5eee7-be26-48c6-b667-f00bed9897a5 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:38.176219 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d17ebe-2009-474d-a87b-2a340c633584 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:38.176424 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d17ebe-2009-474d-a87b-2a340c633584 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:38.176656 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d17ebe-2009-474d-a87b-2a340c633584 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:38.178004 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 164/337] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:31:38.187184 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5eee7-be26-48c6-b667-f00bed9897a5 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:38.189012 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518c39e-d0ac-43f2-a81a-a04f45d33d89 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:38.189526 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518c39e-d0ac-43f2-a81a-a04f45d33d89 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:38.189997 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518c39e-d0ac-43f2-a81a-a04f45d33d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:38.225723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-b518c39e-d0ac-43f2-a81a-a04f45d33d89 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:38.227341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-b518c39e-d0ac-43f2-a81a-a04f45d33d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:38.238082 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd460a-c697-436c-9496-a35281f9be11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:38.239103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd460a-c697-436c-9496-a35281f9be11 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:38.239602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd460a-c697-436c-9496-a35281f9be11 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:38.240066 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd460a-c697-436c-9496-a35281f9be11 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:38.261352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd460a-c697-436c-9496-a35281f9be11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:38.266518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-98fd460a-c697-436c-9496-a35281f9be11 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:38.266518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:38.266518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:38.266518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:38.266518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:38.266518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:38.266518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:38.266518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:38.266518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:38.266518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:38.266518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:38.266518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:38.266518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:38.266518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:38.266518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:38.266518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:38.267515 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:38.267515 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:38.267515 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:38.267515 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:38.267515 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:38.267515 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:38.267515 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:38.267515 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:38.267515 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:38.267515 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:38.267515 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:38.267515 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:38.267515 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:38.267515 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:38.267515 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:38.267515 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:38.268335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:38.268335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:38.268335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:38.268335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:38.268335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:38.268335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:38.268335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:38.268335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:38.268335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:38.268335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:38.268335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:38.268335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:38.273217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 174/338] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Aug 8 16:31:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:38.274853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b09c5-0a06-474f-9bf0-5176db92ff12 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:38.275866 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b09c5-0a06-474f-9bf0-5176db92ff12 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:38.276341 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b09c5-0a06-474f-9bf0-5176db92ff12 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:38.277961 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 175/339] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:38.279279 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:38.279599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411f077-d591-43f1-939c-fae4814d5133 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:38.280077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411f077-d591-43f1-939c-fae4814d5133 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:38.280547 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411f077-d591-43f1-939c-fae4814d5133 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:38.282939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 176/340] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:38.283996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c411f077-d591-43f1-939c-fae4814d5133 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:38.333164 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:38.336161 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 165/341] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:38.337648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86b116-fd74-4d9b-9aa3-e465c71b8645 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:38.338476 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86b116-fd74-4d9b-9aa3-e465c71b8645 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:38.338777 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86b116-fd74-4d9b-9aa3-e465c71b8645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:38.359661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-3f86b116-fd74-4d9b-9aa3-e465c71b8645 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:38.360986 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-3f86b116-fd74-4d9b-9aa3-e465c71b8645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:38.432651 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c411f077-d591-43f1-939c-fae4814d5133 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:38.447016 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 166/342] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:38.448477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6cc05-a5f2-4cec-94a2-84b24467716a None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:38.448757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6cc05-a5f2-4cec-94a2-84b24467716a None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:38.449078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6cc05-a5f2-4cec-94a2-84b24467716a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:38.480051 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c9459-1015-4438-813b-67b2840daf9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:38.481016 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c9459-1015-4438-813b-67b2840daf9a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:38.481513 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c9459-1015-4438-813b-67b2840daf9a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:38.482010 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c9459-1015-4438-813b-67b2840daf9a None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:38.482895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c9459-1015-4438-813b-67b2840daf9a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:38.483417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c9459-1015-4438-813b-67b2840daf9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:38.484996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c9459-1015-4438-813b-67b2840daf9a None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:38.495165 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 177/343] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:31:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:38.496194 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c9459-1015-4438-813b-67b2840daf9a None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:38.596400 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-60a6cc05-a5f2-4cec-94a2-84b24467716a None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:38.598152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-60a6cc05-a5f2-4cec-94a2-84b24467716a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:38.680773 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 167/344] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:38.683075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6cc05-a5f2-4cec-94a2-84b24467716a None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:38.699479 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:38.707379 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c9459-1015-4438-813b-67b2840daf9a None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:38.720583 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:38.764196 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af6166-5f37-4c0a-b2d2-a59975af9f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:38.764844 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af6166-5f37-4c0a-b2d2-a59975af9f88 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:38.765119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af6166-5f37-4c0a-b2d2-a59975af9f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:38.765386 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af6166-5f37-4c0a-b2d2-a59975af9f88 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:38.779990 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af6166-5f37-4c0a-b2d2-a59975af9f88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:38.785209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-b7af6166-5f37-4c0a-b2d2-a59975af9f88 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:38.785209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:38.785209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:38.785209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:38.785209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:38.785209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:38.785209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:38.785209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:38.785209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:38.785209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:38.785209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:38.785209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:38.785209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:38.785209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:38.785209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:38.785209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:38.786153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:38.786153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:38.786153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:38.786153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:38.786153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:38.786153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:38.786153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:38.786153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:38.786153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:38.786153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:38.786153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:38.786153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:38.786153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:38.786153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:38.786153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:38.786153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:38.787409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:38.787409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:38.787409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:38.787409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:38.787409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:38.787409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:38.787409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:38.787409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:38.787409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:38.787409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:38.787409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:38.787409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:38.787409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 178/345] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:38.788206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af6166-5f37-4c0a-b2d2-a59975af9f88 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:38.821340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6cc05-a5f2-4cec-94a2-84b24467716a None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:38.862772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeab668-f53c-4330-9037-f8e7917b52fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:38.863777 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeab668-f53c-4330-9037-f8e7917b52fe None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:38.864385 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeab668-f53c-4330-9037-f8e7917b52fe None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:38.864911 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeab668-f53c-4330-9037-f8e7917b52fe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:38.877226 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeab668-f53c-4330-9037-f8e7917b52fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:38.882913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-6aeab668-f53c-4330-9037-f8e7917b52fe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:31:38.882913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:38.882913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:38.882913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:38.882913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:38.882913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:38.882913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:38.882913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:38.882913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:38.882913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:38.882913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:38.882913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:38.882913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:38.882913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:38.882913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:38.882913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:38.883946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:38.883946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:38.883946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:38.883946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:38.883946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:38.883946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:38.883946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:38.883946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:38.883946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:38.883946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:38.883946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:38.883946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:38.883946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:38.883946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:38.883946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:38.883946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:38.885053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:38.885053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:38.885053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:38.885053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:38.885053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:38.885053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:38.885053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:38.885053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:38.885053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:38.885053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:38.885053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:31:38.885053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:38.887518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 168/346] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:38.888734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeab668-f53c-4330-9037-f8e7917b52fe None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:38.908869 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af6166-5f37-4c0a-b2d2-a59975af9f88 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:38.947015 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4726b-48f2-4e31-88f6-680737ed9676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:38.948267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4726b-48f2-4e31-88f6-680737ed9676 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:38.948686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4726b-48f2-4e31-88f6-680737ed9676 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:38.949027 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4726b-48f2-4e31-88f6-680737ed9676 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:38.976500 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:38.996172 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:39.013889 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeab668-f53c-4330-9037-f8e7917b52fe None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:39.022423 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4726b-48f2-4e31-88f6-680737ed9676 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:39.024374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4726b-48f2-4e31-88f6-680737ed9676 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:39.053620 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182d3f1-0503-4c23-ab90-ce154ea15e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:39.055661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182d3f1-0503-4c23-ab90-ce154ea15e5e None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:39.056091 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182d3f1-0503-4c23-ab90-ce154ea15e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:39.057113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182d3f1-0503-4c23-ab90-ce154ea15e5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:39.069860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182d3f1-0503-4c23-ab90-ce154ea15e5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:39.076412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-1182d3f1-0503-4c23-ab90-ce154ea15e5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:39.076412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:39.076412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:39.076412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:39.076412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:39.076412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:39.076412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:39.076412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:39.076412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:39.076412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:39.076412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:39.076412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:39.076412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:39.076412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:39.076412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:39.076412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:39.077547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:39.077547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:39.077547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:39.077547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:39.077547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:39.077547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:39.077547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:39.077547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:39.077547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:39.077547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:39.077547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:39.077547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:39.077547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:39.077547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:39.077547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:39.077547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:39.078397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:39.078397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:39.078397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:39.078397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:39.078397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:39.078397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:39.078397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:39.078397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:39.078397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:39.078397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:39.078397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:39.078397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:39.080730 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 169/347] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:39.081790 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182d3f1-0503-4c23-ab90-ce154ea15e5e None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:39.141388 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 179/348] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:39.142358 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4726b-48f2-4e31-88f6-680737ed9676 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:39.199748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182d3f1-0503-4c23-ab90-ce154ea15e5e None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:39.205033 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:39.240346 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ac113-d865-4abc-b9cd-e19c9733d0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:39.241263 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ac113-d865-4abc-b9cd-e19c9733d0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:39.241740 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ac113-d865-4abc-b9cd-e19c9733d0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:39.242178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ac113-d865-4abc-b9cd-e19c9733d0d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:39.243071 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ac113-d865-4abc-b9cd-e19c9733d0d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:39.243560 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ac113-d865-4abc-b9cd-e19c9733d0d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:39.245721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ac113-d865-4abc-b9cd-e19c9733d0d1 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:39.255343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4726b-48f2-4e31-88f6-680737ed9676 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:39.255514 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 170/349] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:39.256597 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ac113-d865-4abc-b9cd-e19c9733d0d1 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:39.270433 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:39.299745 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d909536-c4b0-4664-95d4-6ba887b39990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:39.300352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d909536-c4b0-4664-95d4-6ba887b39990 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:39.300697 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d909536-c4b0-4664-95d4-6ba887b39990 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:39.301037 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d909536-c4b0-4664-95d4-6ba887b39990 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:39.301769 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d909536-c4b0-4664-95d4-6ba887b39990 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:39.303170 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d909536-c4b0-4664-95d4-6ba887b39990 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:39.304713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d909536-c4b0-4664-95d4-6ba887b39990 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:39.316103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 180/350] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:39.317176 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d909536-c4b0-4664-95d4-6ba887b39990 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:39.380119 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ac113-d865-4abc-b9cd-e19c9733d0d1 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:39.428977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56db93-f0ea-4113-8559-d5e810755758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:39.429591 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56db93-f0ea-4113-8559-d5e810755758 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:39.429867 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56db93-f0ea-4113-8559-d5e810755758 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:39.430139 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56db93-f0ea-4113-8559-d5e810755758 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:39.430830 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56db93-f0ea-4113-8559-d5e810755758 None admin] RBAC: Adding query filter params (name=member) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:39.431130 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56db93-f0ea-4113-8559-d5e810755758 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:39.433218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56db93-f0ea-4113-8559-d5e810755758 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:39.444245 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 171/351] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:39.446123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56db93-f0ea-4113-8559-d5e810755758 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:39.452708 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d909536-c4b0-4664-95d4-6ba887b39990 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:39.475086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:39.495525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ca1e0-124a-47c9-8263-2b3110b878ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:39.496175 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ca1e0-124a-47c9-8263-2b3110b878ea None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:39.496454 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ca1e0-124a-47c9-8263-2b3110b878ea None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:39.496727 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ca1e0-124a-47c9-8263-2b3110b878ea None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:39.497974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304ca1e0-124a-47c9-8263-2b3110b878ea None admin] RBAC: Authorizing `identity:get_project(project_id=88933471b063476fb90fe841b4e9b75b)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:39.499513 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304ca1e0-124a-47c9-8263-2b3110b878ea None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:39.502393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 181/352] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:39 2022] GET /identity/v3/projects/88933471b063476fb90fe841b4e9b75b => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:39.503125 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ca1e0-124a-47c9-8263-2b3110b878ea None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:39.543466 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:39.585470 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56db93-f0ea-4113-8559-d5e810755758 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:39.633967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85717a8-5b5f-406c-be8e-92fa38cb1a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:39.634933 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85717a8-5b5f-406c-be8e-92fa38cb1a40 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:39.635363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85717a8-5b5f-406c-be8e-92fa38cb1a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:39.635810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85717a8-5b5f-406c-be8e-92fa38cb1a40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:39.636554 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ca1e0-124a-47c9-8263-2b3110b878ea None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:39.684326 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b854e8-dff7-4ba2-9ef5-281cbcbcb50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:39.684987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b854e8-dff7-4ba2-9ef5-281cbcbcb50a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:39.685651 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b854e8-dff7-4ba2-9ef5-281cbcbcb50a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:39.685651 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b854e8-dff7-4ba2-9ef5-281cbcbcb50a None admin] PATH_INFO: `/v3/users/8c47821d07804dd78baadaa7a12b9c48` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:39.687233 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b854e8-dff7-4ba2-9ef5-281cbcbcb50a None admin] RBAC: Authorizing `identity:get_user(user_id=8c47821d07804dd78baadaa7a12b9c48)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:39.688711 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b854e8-dff7-4ba2-9ef5-281cbcbcb50a None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:39.690896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 182/353] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:31:39 2022] GET /identity/v3/users/8c47821d07804dd78baadaa7a12b9c48 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:39.696566 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baaaff2-38aa-4538-ac63-03733fd5436d None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:39.696906 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baaaff2-38aa-4538-ac63-03733fd5436d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:39.697220 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baaaff2-38aa-4538-ac63-03733fd5436d None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:39.698676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 183/354] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:39.699479 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baaaff2-38aa-4538-ac63-03733fd5436d None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:39.755201 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85717a8-5b5f-406c-be8e-92fa38cb1a40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:39.756644 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85717a8-5b5f-406c-be8e-92fa38cb1a40 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:39.785764 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:39.844822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:39.848936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baaaff2-38aa-4538-ac63-03733fd5436d None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:39.887880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 172/355] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:39.888724 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85717a8-5b5f-406c-be8e-92fa38cb1a40 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:39.892335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268d01e-eb68-4929-a133-6f24011e6008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:39.892981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268d01e-eb68-4929-a133-6f24011e6008 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:39.893308 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268d01e-eb68-4929-a133-6f24011e6008 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:39.893530 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268d01e-eb68-4929-a133-6f24011e6008 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:39.977881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268d01e-eb68-4929-a133-6f24011e6008 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:39.979572 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268d01e-eb68-4929-a133-6f24011e6008 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:40.020487 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85717a8-5b5f-406c-be8e-92fa38cb1a40 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:40.047686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:40.060503 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41ef2e-bb3c-436f-8ab4-7d0ad522dad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:40.068077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41ef2e-bb3c-436f-8ab4-7d0ad522dad5 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:40.068077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41ef2e-bb3c-436f-8ab4-7d0ad522dad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:40.068077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41ef2e-bb3c-436f-8ab4-7d0ad522dad5 None admin] PATH_INFO: `/v3/groups/f9901716e50745a2ba3c9edb005aa761` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:40.068077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41ef2e-bb3c-436f-8ab4-7d0ad522dad5 None admin] RBAC: Authorizing `identity:get_group(group_id=f9901716e50745a2ba3c9edb005aa761)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:40.069292 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41ef2e-bb3c-436f-8ab4-7d0ad522dad5 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:40.071089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 173/356] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Aug 8 16:31:39 2022] GET /identity/v3/groups/f9901716e50745a2ba3c9edb005aa761 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:40.072017 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41ef2e-bb3c-436f-8ab4-7d0ad522dad5 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:40.102666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 184/357] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:40.104517 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810297b2-bca3-4496-b2c4-3b36fe47351d None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:40.104794 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810297b2-bca3-4496-b2c4-3b36fe47351d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:40.105065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810297b2-bca3-4496-b2c4-3b36fe47351d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:40.121962 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:40.126217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-810297b2-bca3-4496-b2c4-3b36fe47351d None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:40.129672 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-810297b2-bca3-4496-b2c4-3b36fe47351d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:40.201359 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41ef2e-bb3c-436f-8ab4-7d0ad522dad5 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:40.210755 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 185/358] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:40.212340 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eacbb79-043e-43c7-86e3-09c6a7e1ec89 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:40.212626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eacbb79-043e-43c7-86e3-09c6a7e1ec89 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:40.213422 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eacbb79-043e-43c7-86e3-09c6a7e1ec89 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:40.215054 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 186/359] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:40.215676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eacbb79-043e-43c7-86e3-09c6a7e1ec89 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:40.242413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f546e-bc8e-4494-9402-589e1da677a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:40.242894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f546e-bc8e-4494-9402-589e1da677a9 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:40.243180 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f546e-bc8e-4494-9402-589e1da677a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:40.243446 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f546e-bc8e-4494-9402-589e1da677a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:40.244228 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f546e-bc8e-4494-9402-589e1da677a9 None admin] RBAC: Adding query filter params (role.id=94f7f84c31e0425495dcca9783e7a46e, user.id=8c47821d07804dd78baadaa7a12b9c48, scope.system=all) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:40.244528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f546e-bc8e-4494-9402-589e1da677a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:40.245989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f546e-bc8e-4494-9402-589e1da677a9 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:40.259063 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 174/360] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Aug 8 16:31:40 2022] GET /identity/v3/role_assignments?user.id=8c47821d07804dd78baadaa7a12b9c48&scope.system=all&role.id=94f7f84c31e0425495dcca9783e7a46e => generated 510 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:40.260015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f546e-bc8e-4494-9402-589e1da677a9 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:40.325144 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:40.353834 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eacbb79-043e-43c7-86e3-09c6a7e1ec89 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:40.397227 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:40.403302 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee2546-6dca-4c94-aa46-66a969dde03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:40.405064 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee2546-6dca-4c94-aa46-66a969dde03d None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:40.405064 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee2546-6dca-4c94-aa46-66a969dde03d None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:40.405064 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee2546-6dca-4c94-aa46-66a969dde03d None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:40.406351 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee2546-6dca-4c94-aa46-66a969dde03d None admin] RBAC: Authorizing `identity:get_project(project_id=88933471b063476fb90fe841b4e9b75b)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:40.408101 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee2546-6dca-4c94-aa46-66a969dde03d None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:40.410120 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 187/361] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:40 2022] GET /identity/v3/projects/88933471b063476fb90fe841b4e9b75b => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:40.411322 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee2546-6dca-4c94-aa46-66a969dde03d None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:40.413727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f546e-bc8e-4494-9402-589e1da677a9 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:40.461148 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23fc50-71b2-4423-8172-5053be62b4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:40.461761 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23fc50-71b2-4423-8172-5053be62b4ba None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:40.462037 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23fc50-71b2-4423-8172-5053be62b4ba None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:40.462306 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23fc50-71b2-4423-8172-5053be62b4ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:40.463206 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab23fc50-71b2-4423-8172-5053be62b4ba None admin] RBAC: Adding query filter params (group.id=f9901716e50745a2ba3c9edb005aa761, role.id=fd2599a3f408404599ecea94959efc4a, scope.project.id=88933471b063476fb90fe841b4e9b75b) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:40.463489 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab23fc50-71b2-4423-8172-5053be62b4ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:40.464888 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab23fc50-71b2-4423-8172-5053be62b4ba None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:40.473787 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 175/362] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Aug 8 16:31:40 2022] GET /identity/v3/role_assignments?group.id=f9901716e50745a2ba3c9edb005aa761&scope.project.id=88933471b063476fb90fe841b4e9b75b&role.id=fd2599a3f408404599ecea94959efc4a => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:40.475133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8f4e6-b4b9-4295-ba6b-91ff6edb60f9 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:40.475485 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8f4e6-b4b9-4295-ba6b-91ff6edb60f9 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:40.475828 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8f4e6-b4b9-4295-ba6b-91ff6edb60f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:40.500493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-7bf8f4e6-b4b9-4295-ba6b-91ff6edb60f9 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:40.501732 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-7bf8f4e6-b4b9-4295-ba6b-91ff6edb60f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:40.543193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee2546-6dca-4c94-aa46-66a969dde03d None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:40.588477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ce31d-124d-4f5e-b5c0-8d27ae795645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:40.589430 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ce31d-124d-4f5e-b5c0-8d27ae795645 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:40.589849 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ce31d-124d-4f5e-b5c0-8d27ae795645 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:40.590251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ce31d-124d-4f5e-b5c0-8d27ae795645 None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:40.592071 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ce31d-124d-4f5e-b5c0-8d27ae795645 None admin] RBAC: Authorizing `identity:get_project(project_id=64dad222ddb64408b80bacb84185378f)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:40.594773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ce31d-124d-4f5e-b5c0-8d27ae795645 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:40.596744 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 188/363] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:40 2022] GET /identity/v3/projects/64dad222ddb64408b80bacb84185378f => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:40.601608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ce31d-124d-4f5e-b5c0-8d27ae795645 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:40.601769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 176/364] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:40.601769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8f4e6-b4b9-4295-ba6b-91ff6edb60f9 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:40.636240 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:40.737350 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:40.754517 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8f4e6-b4b9-4295-ba6b-91ff6edb60f9 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:40.757871 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ce31d-124d-4f5e-b5c0-8d27ae795645 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:40.798963 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b3ea2-d971-48cd-8de7-b270568b5c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:40.799617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b3ea2-d971-48cd-8de7-b270568b5c38 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:40.799898 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b3ea2-d971-48cd-8de7-b270568b5c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:40.800168 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b3ea2-d971-48cd-8de7-b270568b5c38 None admin] PATH_INFO: `/v3/groups/f9901716e50745a2ba3c9edb005aa761` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:40.801301 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b3ea2-d971-48cd-8de7-b270568b5c38 None admin] RBAC: Authorizing `identity:get_group(group_id=f9901716e50745a2ba3c9edb005aa761)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:40.801408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff708b9-fad5-4c02-b07c-392d8f2c431a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:40.802005 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff708b9-fad5-4c02-b07c-392d8f2c431a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:40.802290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff708b9-fad5-4c02-b07c-392d8f2c431a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:40.802579 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff708b9-fad5-4c02-b07c-392d8f2c431a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:40.802844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b3ea2-d971-48cd-8de7-b270568b5c38 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:40.805775 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 177/365] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Aug 8 16:31:40 2022] GET /identity/v3/groups/f9901716e50745a2ba3c9edb005aa761 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:40.806836 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b3ea2-d971-48cd-8de7-b270568b5c38 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:40.816285 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff708b9-fad5-4c02-b07c-392d8f2c431a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:40.821536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-4ff708b9-fad5-4c02-b07c-392d8f2c431a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:40.821536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:40.821536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:40.821536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:40.821536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:40.821536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:40.821536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:40.821536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:40.821536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:40.821536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:40.821536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:40.821536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:40.821536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:40.821536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:40.821536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:40.821536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:40.823044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:40.823044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:40.823044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:40.823044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:40.823044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:40.823044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:40.823044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:40.823044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:40.823044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:40.823044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:40.823044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:40.823044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:40.823044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:40.823044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:40.823044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:40.823044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:40.823859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:40.823859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:40.823859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:40.823859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:40.823859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:40.823859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:40.823859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:40.823859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:40.823859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:40.823859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:40.823859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:40.823859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:40.823859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 189/366] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Aug 8 16:31:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:40.824541 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff708b9-fad5-4c02-b07c-392d8f2c431a None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:40.955769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b3ea2-d971-48cd-8de7-b270568b5c38 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:40.956474 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:40.972033 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff708b9-fad5-4c02-b07c-392d8f2c431a None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:41.007698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de4103-2e97-4fee-a384-e25a69ed7f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:41.008409 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de4103-2e97-4fee-a384-e25a69ed7f8d None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:41.008596 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de4103-2e97-4fee-a384-e25a69ed7f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:41.008858 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de4103-2e97-4fee-a384-e25a69ed7f8d None admin] PATH_INFO: `/v3/groups/f9901716e50745a2ba3c9edb005aa761` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:41.010778 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de4103-2e97-4fee-a384-e25a69ed7f8d None admin] RBAC: Authorizing `identity:get_group(group_id=f9901716e50745a2ba3c9edb005aa761)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:41.012447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de4103-2e97-4fee-a384-e25a69ed7f8d None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:41.012813 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114466b6-d17b-4323-a407-8861d03afb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:41.013487 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114466b6-d17b-4323-a407-8861d03afb0c None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:41.013741 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114466b6-d17b-4323-a407-8861d03afb0c None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:41.014249 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114466b6-d17b-4323-a407-8861d03afb0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:41.014366 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 178/367] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Aug 8 16:31:40 2022] GET /identity/v3/groups/f9901716e50745a2ba3c9edb005aa761 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:41.016545 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de4103-2e97-4fee-a384-e25a69ed7f8d None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:41.030745 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114466b6-d17b-4323-a407-8861d03afb0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:41.035901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-114466b6-d17b-4323-a407-8861d03afb0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:31:41.035901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:41.035901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:41.035901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:41.035901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:41.035901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:41.035901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:41.035901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:41.035901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:41.035901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:41.035901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:41.035901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:41.035901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:41.035901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:41.035901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:41.035901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:41.036912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:41.036912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:41.036912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:41.036912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:41.036912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:41.036912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:41.036912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:41.036912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:41.036912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:41.036912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:41.036912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:41.036912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:41.036912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:41.036912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:41.036912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:41.036912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:41.037791 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:41.037791 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:41.037791 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:41.037791 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:41.037791 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:41.037791 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:41.037791 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:41.037791 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:41.037791 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:41.037791 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:41.037791 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:31:41.037791 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:41.040237 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 190/368] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:31:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:41.041360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114466b6-d17b-4323-a407-8861d03afb0c None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:41.053593 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:41.165085 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de4103-2e97-4fee-a384-e25a69ed7f8d None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:41.185142 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114466b6-d17b-4323-a407-8861d03afb0c None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:41.208847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a91eb1-b19a-4aac-a159-a0cce7f74461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:41.209647 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a91eb1-b19a-4aac-a159-a0cce7f74461 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:41.209992 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a91eb1-b19a-4aac-a159-a0cce7f74461 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:41.210349 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a91eb1-b19a-4aac-a159-a0cce7f74461 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:41.211194 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a91eb1-b19a-4aac-a159-a0cce7f74461 None admin] RBAC: Adding query filter params (group.id=f9901716e50745a2ba3c9edb005aa761, role.id=04bf40fcc26347de8167fc215f62c7aa, scope.project.id=88933471b063476fb90fe841b4e9b75b) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:41.211567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a91eb1-b19a-4aac-a159-a0cce7f74461 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:41.214272 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a91eb1-b19a-4aac-a159-a0cce7f74461 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:41.221145 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 179/369] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Aug 8 16:31:41 2022] GET /identity/v3/role_assignments?group.id=f9901716e50745a2ba3c9edb005aa761&scope.project.id=88933471b063476fb90fe841b4e9b75b&role.id=04bf40fcc26347de8167fc215f62c7aa => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:41.222151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a91eb1-b19a-4aac-a159-a0cce7f74461 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:41.227543 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701e12f-4ef5-4897-9abc-6754251b7012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:41.229296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701e12f-4ef5-4897-9abc-6754251b7012 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:41.229774 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701e12f-4ef5-4897-9abc-6754251b7012 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:41.230246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701e12f-4ef5-4897-9abc-6754251b7012 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:41.231172 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701e12f-4ef5-4897-9abc-6754251b7012 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:41.231681 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701e12f-4ef5-4897-9abc-6754251b7012 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:41.233426 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701e12f-4ef5-4897-9abc-6754251b7012 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:41.244633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 191/370] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:31:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:41.246015 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701e12f-4ef5-4897-9abc-6754251b7012 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:41.251620 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:41.342526 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:41.371075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a91eb1-b19a-4aac-a159-a0cce7f74461 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:41.387248 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701e12f-4ef5-4897-9abc-6754251b7012 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:41.415353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce97606-2118-4d8b-9f4d-8af971628d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:41.416997 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce97606-2118-4d8b-9f4d-8af971628d85 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:41.417507 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce97606-2118-4d8b-9f4d-8af971628d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:41.417994 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce97606-2118-4d8b-9f4d-8af971628d85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:41.418983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce97606-2118-4d8b-9f4d-8af971628d85 None admin] RBAC: Adding query filter params (group.id=f9901716e50745a2ba3c9edb005aa761, role.id=04bf40fcc26347de8167fc215f62c7aa, scope.project.id=64dad222ddb64408b80bacb84185378f) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:41.419518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce97606-2118-4d8b-9f4d-8af971628d85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:41.421334 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce97606-2118-4d8b-9f4d-8af971628d85 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:41.428976 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 180/371] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Aug 8 16:31:41 2022] GET /identity/v3/role_assignments?group.id=f9901716e50745a2ba3c9edb005aa761&scope.project.id=64dad222ddb64408b80bacb84185378f&role.id=04bf40fcc26347de8167fc215f62c7aa => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:41.430289 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce97606-2118-4d8b-9f4d-8af971628d85 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:41.441069 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b10b77-8bbd-4269-9a50-7bc0493eeeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:41.442670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b10b77-8bbd-4269-9a50-7bc0493eeeeb None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:41.442954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b10b77-8bbd-4269-9a50-7bc0493eeeeb None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:41.446602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b10b77-8bbd-4269-9a50-7bc0493eeeeb None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:41.446602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b10b77-8bbd-4269-9a50-7bc0493eeeeb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:41.446602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b10b77-8bbd-4269-9a50-7bc0493eeeeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:41.446602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b10b77-8bbd-4269-9a50-7bc0493eeeeb None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:41.460476 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 192/372] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:41.467013 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b10b77-8bbd-4269-9a50-7bc0493eeeeb None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:41.563002 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:41.595598 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce97606-2118-4d8b-9f4d-8af971628d85 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:41.614082 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b10b77-8bbd-4269-9a50-7bc0493eeeeb None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:41.651972 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb1873-ee4f-4797-aca9-211dc29c7e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:41.652886 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb1873-ee4f-4797-aca9-211dc29c7e00 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:41.653176 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb1873-ee4f-4797-aca9-211dc29c7e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:41.653454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb1873-ee4f-4797-aca9-211dc29c7e00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:41.658058 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23dd77a-ccac-41fe-862c-432ffc3730ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:41.659134 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dd77a-ccac-41fe-862c-432ffc3730ae None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:41.659637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dd77a-ccac-41fe-862c-432ffc3730ae None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:41.660204 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dd77a-ccac-41fe-862c-432ffc3730ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:41.679559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:41.751650 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb1873-ee4f-4797-aca9-211dc29c7e00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:41.753997 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb1873-ee4f-4797-aca9-211dc29c7e00 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:41.784702 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23dd77a-ccac-41fe-862c-432ffc3730ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:41.786173 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23dd77a-ccac-41fe-862c-432ffc3730ae None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:41.865311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:41.913576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 193/373] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:41.920868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23dd77a-ccac-41fe-862c-432ffc3730ae None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:41.985148 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 181/374] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:41.991069 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb1873-ee4f-4797-aca9-211dc29c7e00 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:42.040417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23dd77a-ccac-41fe-862c-432ffc3730ae None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:42.068774 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:42.079093 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae5096-74a5-4802-9d4d-03a347722d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:42.084195 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae5096-74a5-4802-9d4d-03a347722d41 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:42.084195 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae5096-74a5-4802-9d4d-03a347722d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:42.084195 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae5096-74a5-4802-9d4d-03a347722d41 None admin] PATH_INFO: `/v3/projects/3a8d1eb28476411285e5ac5c0ab9ab53` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:42.084195 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae5096-74a5-4802-9d4d-03a347722d41 None admin] RBAC: Authorizing `identity:get_project(project_id=3a8d1eb28476411285e5ac5c0ab9ab53)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:42.084195 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae5096-74a5-4802-9d4d-03a347722d41 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:42.084922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 194/375] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:41 2022] GET /identity/v3/projects/3a8d1eb28476411285e5ac5c0ab9ab53 => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:42.091401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae5096-74a5-4802-9d4d-03a347722d41 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:42.152170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb1873-ee4f-4797-aca9-211dc29c7e00 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:42.165650 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:42.231562 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4acc6-9ac1-4e28-b617-463cc68b7d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:42.232205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4acc6-9ac1-4e28-b617-463cc68b7d57 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:42.232479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4acc6-9ac1-4e28-b617-463cc68b7d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:42.232741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4acc6-9ac1-4e28-b617-463cc68b7d57 None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:42.234002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af4acc6-9ac1-4e28-b617-463cc68b7d57 None admin] RBAC: Authorizing `identity:get_project(project_id=64dad222ddb64408b80bacb84185378f)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:42.235477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af4acc6-9ac1-4e28-b617-463cc68b7d57 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:42.237166 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 182/376] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:41 2022] GET /identity/v3/projects/64dad222ddb64408b80bacb84185378f => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:42.243329 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4acc6-9ac1-4e28-b617-463cc68b7d57 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:42.281580 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae5096-74a5-4802-9d4d-03a347722d41 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:42.349713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8b870-65a6-4dc7-9cff-ca69d9e0b711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:42.350400 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8b870-65a6-4dc7-9cff-ca69d9e0b711 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:42.350698 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8b870-65a6-4dc7-9cff-ca69d9e0b711 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:42.350963 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8b870-65a6-4dc7-9cff-ca69d9e0b711 None admin] PATH_INFO: `/v3/groups/8f09983cdd674c14abd3004382998b60` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:42.362849 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d8b870-65a6-4dc7-9cff-ca69d9e0b711 None admin] RBAC: Authorizing `identity:get_group(group_id=8f09983cdd674c14abd3004382998b60)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:42.364325 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d8b870-65a6-4dc7-9cff-ca69d9e0b711 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:42.366902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 195/377] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Aug 8 16:31:42 2022] GET /identity/v3/groups/8f09983cdd674c14abd3004382998b60 => generated 229 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:42.373261 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8b870-65a6-4dc7-9cff-ca69d9e0b711 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:42.435296 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4acc6-9ac1-4e28-b617-463cc68b7d57 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:42.477640 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:42.520811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ccbd8d-e635-485b-896f-20d977aa6cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:42.521467 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ccbd8d-e635-485b-896f-20d977aa6cff None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:42.521744 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ccbd8d-e635-485b-896f-20d977aa6cff None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:42.522011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ccbd8d-e635-485b-896f-20d977aa6cff None admin] PATH_INFO: `/v3/groups/f9901716e50745a2ba3c9edb005aa761` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:42.523237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ccbd8d-e635-485b-896f-20d977aa6cff None admin] RBAC: Authorizing `identity:get_group(group_id=f9901716e50745a2ba3c9edb005aa761)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:42.524760 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ccbd8d-e635-485b-896f-20d977aa6cff None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:42.528404 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 183/378] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Aug 8 16:31:42 2022] GET /identity/v3/groups/f9901716e50745a2ba3c9edb005aa761 => generated 226 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:42.533059 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ccbd8d-e635-485b-896f-20d977aa6cff None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:42.566859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:42.577524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8b870-65a6-4dc7-9cff-ca69d9e0b711 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:42.621039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7fc10-4539-4d67-a430-0dbce8ce47c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:42.622077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7fc10-4539-4d67-a430-0dbce8ce47c4 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:42.622595 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7fc10-4539-4d67-a430-0dbce8ce47c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:42.623024 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7fc10-4539-4d67-a430-0dbce8ce47c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:42.624261 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7fc10-4539-4d67-a430-0dbce8ce47c4 None admin] RBAC: Adding query filter params (group.id=8f09983cdd674c14abd3004382998b60, role.id=e578ef831127463d9b48cc89fe20329c, scope.project.id=3a8d1eb28476411285e5ac5c0ab9ab53) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:42.630591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7fc10-4539-4d67-a430-0dbce8ce47c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:42.630591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7fc10-4539-4d67-a430-0dbce8ce47c4 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:42.635327 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 196/379] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Aug 8 16:31:42 2022] GET /identity/v3/role_assignments?group.id=8f09983cdd674c14abd3004382998b60&scope.project.id=3a8d1eb28476411285e5ac5c0ab9ab53&role.id=e578ef831127463d9b48cc89fe20329c => generated 260 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:42.730678 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ccbd8d-e635-485b-896f-20d977aa6cff None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:42.770798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc56b6a-712f-4cd4-bf97-acfff7411ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:42.771820 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc56b6a-712f-4cd4-bf97-acfff7411ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:42.772310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc56b6a-712f-4cd4-bf97-acfff7411ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:42.772825 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc56b6a-712f-4cd4-bf97-acfff7411ab0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:42.773796 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc56b6a-712f-4cd4-bf97-acfff7411ab0 None admin] RBAC: Adding query filter params (group.id=f9901716e50745a2ba3c9edb005aa761, role.id=fd2599a3f408404599ecea94959efc4a, scope.project.id=64dad222ddb64408b80bacb84185378f) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:42.774348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc56b6a-712f-4cd4-bf97-acfff7411ab0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:42.777100 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc56b6a-712f-4cd4-bf97-acfff7411ab0 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:42.784176 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 184/380] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Aug 8 16:31:42 2022] GET /identity/v3/role_assignments?group.id=f9901716e50745a2ba3c9edb005aa761&scope.project.id=64dad222ddb64408b80bacb84185378f&role.id=fd2599a3f408404599ecea94959efc4a => generated 260 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:42.802146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2035be-57c2-41b9-a626-1e4bbf2f842b None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:42.802465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2035be-57c2-41b9-a626-1e4bbf2f842b None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:42.802693 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2035be-57c2-41b9-a626-1e4bbf2f842b None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:42.804076 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 197/381] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:42.810494 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9065aa3-2af7-4fab-92ca-8d1bb0e31250 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:42.811146 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9065aa3-2af7-4fab-92ca-8d1bb0e31250 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:42.811636 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9065aa3-2af7-4fab-92ca-8d1bb0e31250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:42.845347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-a9065aa3-2af7-4fab-92ca-8d1bb0e31250 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:42.847023 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-a9065aa3-2af7-4fab-92ca-8d1bb0e31250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:42.914433 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:42.969501 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 185/382] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 08 16:31:43.004617 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2035be-57c2-41b9-a626-1e4bbf2f842b None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:43.160089 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2035be-57c2-41b9-a626-1e4bbf2f842b None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:43.215956 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3f169-30e7-4813-a77d-a71e946819ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:43.216532 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3f169-30e7-4813-a77d-a71e946819ab None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:43.216806 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3f169-30e7-4813-a77d-a71e946819ab None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:43.217074 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3f169-30e7-4813-a77d-a71e946819ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:43.239564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3f169-30e7-4813-a77d-a71e946819ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:43.245450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-a1e3f169-30e7-4813-a77d-a71e946819ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:43.245450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:43.245450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:43.245450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:43.245450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:43.245450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:43.245450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:43.245450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:43.245450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:43.245450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:43.245450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:43.245450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:43.245450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:43.245450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:43.245450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:43.245450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:43.246417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:43.246417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:43.246417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:43.246417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:43.246417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:43.246417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:43.246417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:43.246417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:43.246417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:43.246417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:43.246417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:43.246417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:43.246417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:43.246417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:43.246417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:43.246417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:43.247296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:43.247296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:43.247296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:43.247296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:43.247296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:43.247296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:43.247296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:43.247296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:43.247296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:43.247296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:43.247296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:43.247296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:43.247296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 198/383] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Aug 8 16:31:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:43.252624 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9065aa3-2af7-4fab-92ca-8d1bb0e31250 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:43.397732 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9065aa3-2af7-4fab-92ca-8d1bb0e31250 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:43.439351 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db4521-627d-4b78-b078-4926211c978e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:43.440041 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db4521-627d-4b78-b078-4926211c978e None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:43.440328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db4521-627d-4b78-b078-4926211c978e None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:43.440602 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db4521-627d-4b78-b078-4926211c978e None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:43.441282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db4521-627d-4b78-b078-4926211c978e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:43.441584 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db4521-627d-4b78-b078-4926211c978e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:43.443006 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db4521-627d-4b78-b078-4926211c978e None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:43.459067 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 186/384] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:31:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:43.468205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3f169-30e7-4813-a77d-a71e946819ab None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:43.505181 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:43.623589 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3f169-30e7-4813-a77d-a71e946819ab None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:43.672184 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae04681-a983-4c8b-a659-5d1f83efcc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:43.672821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae04681-a983-4c8b-a659-5d1f83efcc41 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:43.673098 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae04681-a983-4c8b-a659-5d1f83efcc41 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:43.673368 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae04681-a983-4c8b-a659-5d1f83efcc41 None admin] PATH_INFO: `/v3/groups/f9901716e50745a2ba3c9edb005aa761` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:43.674481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae04681-a983-4c8b-a659-5d1f83efcc41 None admin] RBAC: Authorizing `identity:get_group(group_id=f9901716e50745a2ba3c9edb005aa761)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:43.675949 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae04681-a983-4c8b-a659-5d1f83efcc41 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:43.677825 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 199/385] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Aug 8 16:31:43 2022] GET /identity/v3/groups/f9901716e50745a2ba3c9edb005aa761 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:43.684463 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db4521-627d-4b78-b078-4926211c978e None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:43.768723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:43.845734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db4521-627d-4b78-b078-4926211c978e None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:43.881131 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4579e-c8f2-47dd-911a-6f9d4939fb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:43.881768 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4579e-c8f2-47dd-911a-6f9d4939fb1b None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:43.882093 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4579e-c8f2-47dd-911a-6f9d4939fb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:43.882365 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4579e-c8f2-47dd-911a-6f9d4939fb1b None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:43.883540 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf4579e-c8f2-47dd-911a-6f9d4939fb1b None admin] RBAC: Authorizing `identity:get_project(project_id=88933471b063476fb90fe841b4e9b75b)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:43.886156 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf4579e-c8f2-47dd-911a-6f9d4939fb1b None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:43.888208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 187/386] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:43 2022] GET /identity/v3/projects/88933471b063476fb90fe841b4e9b75b => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:43.894001 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae04681-a983-4c8b-a659-5d1f83efcc41 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:44.036808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:44.054626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae04681-a983-4c8b-a659-5d1f83efcc41 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:44.099626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb002d2-88d4-4244-85a1-d266907dfb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:44.100294 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb002d2-88d4-4244-85a1-d266907dfb6e None admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:44.100567 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb002d2-88d4-4244-85a1-d266907dfb6e None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:44.100838 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb002d2-88d4-4244-85a1-d266907dfb6e None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b/groups/f9901716e50745a2ba3c9edb005aa761/roles/fd2599a3f408404599ecea94959efc4a` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:44.103042 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb002d2-88d4-4244-85a1-d266907dfb6e None admin] RBAC: Authorizing `identity:create_grant(project_id=88933471b063476fb90fe841b4e9b75b, group_id=f9901716e50745a2ba3c9edb005aa761, role_id=fd2599a3f408404599ecea94959efc4a)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:44.104526 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb002d2-88d4-4244-85a1-d266907dfb6e None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:44.160239 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe533e79-cbbf-4028-adce-9c09d1f22f37 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:44.160541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe533e79-cbbf-4028-adce-9c09d1f22f37 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:44.160903 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe533e79-cbbf-4028-adce-9c09d1f22f37 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:44.162686 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 188/387] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:31:44.167982 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920710da-a6f6-468b-84ee-bc5e3fa7e6d0 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:44.168270 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920710da-a6f6-468b-84ee-bc5e3fa7e6d0 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:44.168599 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920710da-a6f6-468b-84ee-bc5e3fa7e6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:44.191647 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-920710da-a6f6-468b-84ee-bc5e3fa7e6d0 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:44.194119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 200/388] 10.222.0.13 () {58 vars in 1442 bytes} [Mon Aug 8 16:31:43 2022] PUT /identity/v3/projects/88933471b063476fb90fe841b4e9b75b/groups/f9901716e50745a2ba3c9edb005aa761/roles/fd2599a3f408404599ecea94959efc4a => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:31:44.195270 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-920710da-a6f6-468b-84ee-bc5e3fa7e6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:44.287966 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 189/389] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:44.324542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb002d2-88d4-4244-85a1-d266907dfb6e None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:44.362153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bacc573-9938-46ff-9e4d-2b3af46b6df6 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:44.362462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bacc573-9938-46ff-9e4d-2b3af46b6df6 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:44.362721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bacc573-9938-46ff-9e4d-2b3af46b6df6 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:44.364318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 190/390] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:31:44.371135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1ea0e-3b24-454a-8544-b3ead5998bd6 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:44.371415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1ea0e-3b24-454a-8544-b3ead5998bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:44.371878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1ea0e-3b24-454a-8544-b3ead5998bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:44.391039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-d2b1ea0e-3b24-454a-8544-b3ead5998bd6 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:44.392613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d2b1ea0e-3b24-454a-8544-b3ead5998bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:44.465545 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:44.506050 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb002d2-88d4-4244-85a1-d266907dfb6e None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:44.517530 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 191/391] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:31:44.547984 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1ea0e-3b24-454a-8544-b3ead5998bd6 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:44.549684 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc94d4-a816-4ea2-8f96-bff11df52ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:44.550805 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc94d4-a816-4ea2-8f96-bff11df52ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:44.552186 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc94d4-a816-4ea2-8f96-bff11df52ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:44.552664 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc94d4-a816-4ea2-8f96-bff11df52ba5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:44.569480 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddc94d4-a816-4ea2-8f96-bff11df52ba5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:44.576246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-7ddc94d4-a816-4ea2-8f96-bff11df52ba5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:44.576246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:44.576246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:44.576246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:44.576246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:44.576246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:44.576246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:44.576246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:44.576246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:44.576246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:44.576246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:44.576246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:44.576246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:44.576246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:44.576246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:44.576246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:44.577189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:44.577189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:44.577189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:44.577189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:44.577189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:44.577189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:44.577189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:44.577189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:44.577189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:44.577189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:44.577189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:44.577189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:44.577189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:44.577189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:44.577189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:44.577189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:44.578004 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:44.578004 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:44.578004 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:44.578004 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:44.578004 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:44.578004 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:44.578004 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:44.578004 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:44.578004 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:44.578004 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:44.578004 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:44.578004 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:44.580476 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 201/392] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:44.586367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc94d4-a816-4ea2-8f96-bff11df52ba5 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:44.599777 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:44.678187 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1ea0e-3b24-454a-8544-b3ead5998bd6 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:44.706112 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc94d4-a816-4ea2-8f96-bff11df52ba5 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:44.736869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47b771-2d15-4e16-962a-6ce7dad80af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:44.737534 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47b771-2d15-4e16-962a-6ce7dad80af2 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:44.737811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47b771-2d15-4e16-962a-6ce7dad80af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:44.738081 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47b771-2d15-4e16-962a-6ce7dad80af2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:44.763222 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad47b771-2d15-4e16-962a-6ce7dad80af2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:44.770253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-ad47b771-2d15-4e16-962a-6ce7dad80af2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:44.770253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:44.770253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:44.770253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:44.770253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:44.770253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:44.770253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:44.770253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:44.770253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:44.770253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:44.770253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:44.770253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:44.770253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:44.770253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:44.770253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:44.770253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:44.771328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:44.771328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:44.771328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:44.771328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:44.771328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:44.771328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:44.771328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:44.771328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:44.771328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:44.771328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:44.771328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:44.771328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:44.771328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:44.771328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:44.771328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:44.771328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:44.772155 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:44.772155 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:44.772155 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:44.772155 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:44.772155 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:44.772155 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:44.772155 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:44.772155 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:44.772155 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:44.772155 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:44.772155 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:44.772155 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:44.783727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 192/393] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:31:44.787227 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47b771-2d15-4e16-962a-6ce7dad80af2 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:44.791864 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34905f1-40ab-4668-a58b-a1d946aa6536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:44.792536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34905f1-40ab-4668-a58b-a1d946aa6536 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:44.792813 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34905f1-40ab-4668-a58b-a1d946aa6536 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:44.793080 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34905f1-40ab-4668-a58b-a1d946aa6536 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:44.793765 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34905f1-40ab-4668-a58b-a1d946aa6536 None admin] RBAC: Adding query filter params (name=member) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:44.794062 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34905f1-40ab-4668-a58b-a1d946aa6536 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:44.795504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34905f1-40ab-4668-a58b-a1d946aa6536 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:44.822765 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 202/394] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:44.828793 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34905f1-40ab-4668-a58b-a1d946aa6536 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:44.860737 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:44.916605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:44.986783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34905f1-40ab-4668-a58b-a1d946aa6536 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:45.000632 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47b771-2d15-4e16-962a-6ce7dad80af2 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:45.029158 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296dcc5-1ed6-4db7-9e01-a2253fb85a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:45.029865 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296dcc5-1ed6-4db7-9e01-a2253fb85a56 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:45.030256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296dcc5-1ed6-4db7-9e01-a2253fb85a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:45.030537 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296dcc5-1ed6-4db7-9e01-a2253fb85a56 None admin] PATH_INFO: `/v3/groups/f9901716e50745a2ba3c9edb005aa761` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:45.031886 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296dcc5-1ed6-4db7-9e01-a2253fb85a56 None admin] RBAC: Authorizing `identity:get_group(group_id=f9901716e50745a2ba3c9edb005aa761)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:45.033546 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296dcc5-1ed6-4db7-9e01-a2253fb85a56 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:45.035916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 203/395] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Aug 8 16:31:44 2022] GET /identity/v3/groups/f9901716e50745a2ba3c9edb005aa761 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:45.037403 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87b641-d551-406c-bd1c-cb47334ad652 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:45.037693 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87b641-d551-406c-bd1c-cb47334ad652 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:45.037965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87b641-d551-406c-bd1c-cb47334ad652 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:45.039932 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 204/396] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:45.043418 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87b641-d551-406c-bd1c-cb47334ad652 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:45.047319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6b24f-9d67-41a6-ae7c-dac94326222d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:45.048897 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6b24f-9d67-41a6-ae7c-dac94326222d None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:45.049183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6b24f-9d67-41a6-ae7c-dac94326222d None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:45.049456 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6b24f-9d67-41a6-ae7c-dac94326222d None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:45.050147 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d6b24f-9d67-41a6-ae7c-dac94326222d None admin] RBAC: Adding query filter params (name=member) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:45.050451 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d6b24f-9d67-41a6-ae7c-dac94326222d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:45.051961 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d6b24f-9d67-41a6-ae7c-dac94326222d None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:45.064757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 193/397] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:45.066102 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f0df5-f0c0-4feb-a23d-44ba5bd0bf55 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:45.066638 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f0df5-f0c0-4feb-a23d-44ba5bd0bf55 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:45.066931 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f0df5-f0c0-4feb-a23d-44ba5bd0bf55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:45.092055 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-0b7f0df5-f0c0-4feb-a23d-44ba5bd0bf55 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:45.093428 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-0b7f0df5-f0c0-4feb-a23d-44ba5bd0bf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:45.178741 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87b641-d551-406c-bd1c-cb47334ad652 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:45.179464 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 194/398] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:45.180557 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f0df5-f0c0-4feb-a23d-44ba5bd0bf55 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:45.208892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:45.224745 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17760592-422a-4d4b-8c39-257237c8b68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:45.225429 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17760592-422a-4d4b-8c39-257237c8b68a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:45.225656 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17760592-422a-4d4b-8c39-257237c8b68a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:45.225927 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17760592-422a-4d4b-8c39-257237c8b68a None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:45.227139 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17760592-422a-4d4b-8c39-257237c8b68a None admin] RBAC: Authorizing `identity:get_project(project_id=88933471b063476fb90fe841b4e9b75b)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:45.229994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17760592-422a-4d4b-8c39-257237c8b68a None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:45.232461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 205/399] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:45 2022] GET /identity/v3/projects/88933471b063476fb90fe841b4e9b75b => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:45.233666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17760592-422a-4d4b-8c39-257237c8b68a None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:45.246914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:45.320768 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f0df5-f0c0-4feb-a23d-44ba5bd0bf55 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:45.365947 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fddaea-ba2d-4a1f-84c0-567a7e2991e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:45.366635 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fddaea-ba2d-4a1f-84c0-567a7e2991e6 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:45.367164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fddaea-ba2d-4a1f-84c0-567a7e2991e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:45.367443 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fddaea-ba2d-4a1f-84c0-567a7e2991e6 None admin] PATH_INFO: `/v3/groups/f9901716e50745a2ba3c9edb005aa761` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:45.369398 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fddaea-ba2d-4a1f-84c0-567a7e2991e6 None admin] RBAC: Authorizing `identity:get_group(group_id=f9901716e50745a2ba3c9edb005aa761)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:45.371254 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fddaea-ba2d-4a1f-84c0-567a7e2991e6 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:45.373967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 195/400] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Aug 8 16:31:45 2022] GET /identity/v3/groups/f9901716e50745a2ba3c9edb005aa761 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:45.374854 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fddaea-ba2d-4a1f-84c0-567a7e2991e6 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:45.375228 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17760592-422a-4d4b-8c39-257237c8b68a None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:45.419210 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714293c-ea6b-4cec-92e2-c06f5fc9990e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:45.426163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714293c-ea6b-4cec-92e2-c06f5fc9990e None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:45.426163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714293c-ea6b-4cec-92e2-c06f5fc9990e None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:45.426163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714293c-ea6b-4cec-92e2-c06f5fc9990e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:45.440897 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2714293c-ea6b-4cec-92e2-c06f5fc9990e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:45.448153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-2714293c-ea6b-4cec-92e2-c06f5fc9990e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:31:45.448153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:45.448153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:45.448153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:45.448153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:45.448153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:45.448153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:45.448153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:45.448153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:45.448153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:45.448153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:45.448153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:45.448153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:45.448153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:45.448153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:45.448153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:45.449438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:45.449438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:45.449438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:45.449438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:45.449438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:45.449438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:45.449438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:45.449438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:45.449438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:45.449438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:45.449438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:45.449438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:45.449438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:45.449438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:45.449438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:45.449438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:45.450509 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:45.450509 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:45.450509 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:45.450509 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:45.450509 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:45.450509 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:45.450509 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:45.450509 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:45.450509 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:45.450509 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:45.450509 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:31:45.450509 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:45.452471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 206/401] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:31:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:45.453480 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714293c-ea6b-4cec-92e2-c06f5fc9990e None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:45.507104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:45.520716 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fddaea-ba2d-4a1f-84c0-567a7e2991e6 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:45.535672 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:45.569721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e91b8-2315-4d14-9f34-9273f63dc92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:45.570918 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e91b8-2315-4d14-9f34-9273f63dc92f None admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:45.571672 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e91b8-2315-4d14-9f34-9273f63dc92f None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:45.572960 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e91b8-2315-4d14-9f34-9273f63dc92f None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b/groups/f9901716e50745a2ba3c9edb005aa761/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:45.575302 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714293c-ea6b-4cec-92e2-c06f5fc9990e None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:45.575422 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e91b8-2315-4d14-9f34-9273f63dc92f None admin] RBAC: Authorizing `identity:create_grant(project_id=88933471b063476fb90fe841b4e9b75b, group_id=f9901716e50745a2ba3c9edb005aa761, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:45.577213 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e91b8-2315-4d14-9f34-9273f63dc92f None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:45.618508 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed39654-09d3-439e-8b7c-15313ddb510f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:45.619625 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed39654-09d3-439e-8b7c-15313ddb510f None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:45.619913 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed39654-09d3-439e-8b7c-15313ddb510f None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:45.620193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed39654-09d3-439e-8b7c-15313ddb510f None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:45.621466 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed39654-09d3-439e-8b7c-15313ddb510f None admin] RBAC: Authorizing `identity:get_project(project_id=64dad222ddb64408b80bacb84185378f)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:45.623075 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed39654-09d3-439e-8b7c-15313ddb510f None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:45.624808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 207/402] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:45 2022] GET /identity/v3/projects/64dad222ddb64408b80bacb84185378f => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:45.625583 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed39654-09d3-439e-8b7c-15313ddb510f None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:45.660378 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 196/403] 10.222.0.13 () {58 vars in 1442 bytes} [Mon Aug 8 16:31:45 2022] PUT /identity/v3/projects/88933471b063476fb90fe841b4e9b75b/groups/f9901716e50745a2ba3c9edb005aa761/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:31:45.661371 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e91b8-2315-4d14-9f34-9273f63dc92f None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:45.770065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed39654-09d3-439e-8b7c-15313ddb510f None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:45.811798 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c826894-467f-4826-af3e-b309b673b631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:45.812088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e91b8-2315-4d14-9f34-9273f63dc92f None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:45.812915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c826894-467f-4826-af3e-b309b673b631 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:45.813390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c826894-467f-4826-af3e-b309b673b631 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:45.813845 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c826894-467f-4826-af3e-b309b673b631 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:45.814795 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c826894-467f-4826-af3e-b309b673b631 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:45.815289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c826894-467f-4826-af3e-b309b673b631 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:45.816914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c826894-467f-4826-af3e-b309b673b631 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:45.819065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:45.829093 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 208/404] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:45.832963 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e207057-790e-4937-b72b-5c60385b61bd None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:45.832963 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e207057-790e-4937-b72b-5c60385b61bd None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:45.832963 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e207057-790e-4937-b72b-5c60385b61bd None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:45.832963 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 209/405] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:45.840575 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e207057-790e-4937-b72b-5c60385b61bd None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:45.874126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d803b-6120-4712-b294-50770edcf981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:45.874730 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d803b-6120-4712-b294-50770edcf981 None admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:45.874998 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d803b-6120-4712-b294-50770edcf981 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:45.875263 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d803b-6120-4712-b294-50770edcf981 None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f/groups/f9901716e50745a2ba3c9edb005aa761/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:45.878364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d803b-6120-4712-b294-50770edcf981 None admin] RBAC: Authorizing `identity:create_grant(project_id=64dad222ddb64408b80bacb84185378f, group_id=f9901716e50745a2ba3c9edb005aa761, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:45.879741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d803b-6120-4712-b294-50770edcf981 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:45.969256 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 197/406] 10.222.0.13 () {58 vars in 1442 bytes} [Mon Aug 8 16:31:45 2022] PUT /identity/v3/projects/64dad222ddb64408b80bacb84185378f/groups/f9901716e50745a2ba3c9edb005aa761/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:31:45.976465 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34875c13-925c-4e8e-b931-b1e10dd63b75 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:45.976915 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34875c13-925c-4e8e-b931-b1e10dd63b75 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:45.977369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34875c13-925c-4e8e-b931-b1e10dd63b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:46.008868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e207057-790e-4937-b72b-5c60385b61bd None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:46.037873 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-34875c13-925c-4e8e-b931-b1e10dd63b75 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:46.040051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-34875c13-925c-4e8e-b931-b1e10dd63b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:46.054755 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:46.080696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351b670-0c23-4201-aff6-0a6415ebd226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:46.082109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351b670-0c23-4201-aff6-0a6415ebd226 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:46.083160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351b670-0c23-4201-aff6-0a6415ebd226 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:46.083829 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351b670-0c23-4201-aff6-0a6415ebd226 None admin] PATH_INFO: `/v3/groups/8f09983cdd674c14abd3004382998b60` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:46.086109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351b670-0c23-4201-aff6-0a6415ebd226 None admin] RBAC: Authorizing `identity:get_group(group_id=8f09983cdd674c14abd3004382998b60)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:46.088933 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351b670-0c23-4201-aff6-0a6415ebd226 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:46.093734 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 210/407] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Aug 8 16:31:45 2022] GET /identity/v3/groups/8f09983cdd674c14abd3004382998b60 => generated 229 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:46.107631 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351b670-0c23-4201-aff6-0a6415ebd226 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:46.203508 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:46.233876 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 198/408] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:46.253232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351b670-0c23-4201-aff6-0a6415ebd226 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:46.268788 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34875c13-925c-4e8e-b931-b1e10dd63b75 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:46.304876 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c7445-09a9-4b06-8cfc-759f1d559189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:46.305544 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c7445-09a9-4b06-8cfc-759f1d559189 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:46.305821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c7445-09a9-4b06-8cfc-759f1d559189 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:46.306182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c7445-09a9-4b06-8cfc-759f1d559189 None admin] PATH_INFO: `/v3/projects/3a8d1eb28476411285e5ac5c0ab9ab53` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:46.309081 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c7445-09a9-4b06-8cfc-759f1d559189 None admin] RBAC: Authorizing `identity:get_project(project_id=3a8d1eb28476411285e5ac5c0ab9ab53)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:46.310870 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c7445-09a9-4b06-8cfc-759f1d559189 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:46.313846 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 211/409] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:46 2022] GET /identity/v3/projects/3a8d1eb28476411285e5ac5c0ab9ab53 => generated 344 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:46.320717 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c7445-09a9-4b06-8cfc-759f1d559189 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:46.462201 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34875c13-925c-4e8e-b931-b1e10dd63b75 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:46.536640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c7445-09a9-4b06-8cfc-759f1d559189 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:46.552065 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a3f8a-3c69-4e8b-8d51-2cde8320a48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:46.553026 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a3f8a-3c69-4e8b-8d51-2cde8320a48a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:46.553545 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a3f8a-3c69-4e8b-8d51-2cde8320a48a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:46.553957 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a3f8a-3c69-4e8b-8d51-2cde8320a48a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:46.567551 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:46.575118 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a3f8a-3c69-4e8b-8d51-2cde8320a48a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:46.584651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-cd0a3f8a-3c69-4e8b-8d51-2cde8320a48a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:46.584651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:46.584651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:46.584651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:46.584651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:46.584651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:46.584651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:46.584651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:46.584651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:46.584651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:46.584651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:46.584651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:46.584651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:46.584651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:46.584651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:46.584651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:46.586394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:46.586394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:46.586394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:46.586394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:46.586394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:46.586394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:46.586394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:46.586394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:46.586394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:46.586394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:46.586394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:46.586394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:46.586394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:46.586394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:46.586394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:46.586394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:46.590135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:46.590135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:46.590135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:46.590135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:46.590135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:46.590135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:46.590135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:46.590135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:46.590135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:46.590135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:46.590135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:46.590135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:46.590135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 199/410] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Aug 8 16:31:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:46.598738 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:46.599041 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a3f8a-3c69-4e8b-8d51-2cde8320a48a None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:46.611919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea836c-95c4-4b9b-930a-1b92b38f3a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:46.612654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea836c-95c4-4b9b-930a-1b92b38f3a4c None admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:46.613026 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea836c-95c4-4b9b-930a-1b92b38f3a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:46.613307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea836c-95c4-4b9b-930a-1b92b38f3a4c None admin] PATH_INFO: `/v3/projects/3a8d1eb28476411285e5ac5c0ab9ab53/groups/8f09983cdd674c14abd3004382998b60/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:46.616453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea836c-95c4-4b9b-930a-1b92b38f3a4c None admin] RBAC: Authorizing `identity:create_grant(project_id=3a8d1eb28476411285e5ac5c0ab9ab53, group_id=8f09983cdd674c14abd3004382998b60, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:46.619919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea836c-95c4-4b9b-930a-1b92b38f3a4c None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:46.702672 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 212/411] 10.222.0.13 () {58 vars in 1442 bytes} [Mon Aug 8 16:31:46 2022] PUT /identity/v3/projects/3a8d1eb28476411285e5ac5c0ab9ab53/groups/8f09983cdd674c14abd3004382998b60/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:31:46.704328 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585b9a0-d2a1-4dc7-9e64-a4fb820771a1 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:46.704873 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585b9a0-d2a1-4dc7-9e64-a4fb820771a1 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:46.705351 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585b9a0-d2a1-4dc7-9e64-a4fb820771a1 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:46.708259 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 213/412] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:46.719756 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404fdc4-2bec-4036-9b93-5b5213fe0b3f None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:46.721305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404fdc4-2bec-4036-9b93-5b5213fe0b3f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:46.721799 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404fdc4-2bec-4036-9b93-5b5213fe0b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:46.757696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-9404fdc4-2bec-4036-9b93-5b5213fe0b3f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:46.760419 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-9404fdc4-2bec-4036-9b93-5b5213fe0b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:46.761784 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a3f8a-3c69-4e8b-8d51-2cde8320a48a None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:46.843450 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2389e1-ded4-4521-a540-1582d67b3e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:46.843813 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2389e1-ded4-4521-a540-1582d67b3e1a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:46.844069 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2389e1-ded4-4521-a540-1582d67b3e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:46.844339 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2389e1-ded4-4521-a540-1582d67b3e1a None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:46.845876 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2389e1-ded4-4521-a540-1582d67b3e1a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:46.846182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2389e1-ded4-4521-a540-1582d67b3e1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:46.847931 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2389e1-ded4-4521-a540-1582d67b3e1a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:46.862823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 214/413] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:46.866444 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 200/414] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:31:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:46.873428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404fdc4-2bec-4036-9b93-5b5213fe0b3f None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:46.899872 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2389e1-ded4-4521-a540-1582d67b3e1a None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:47.003247 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:47.010612 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404fdc4-2bec-4036-9b93-5b5213fe0b3f None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:47.038664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2389e1-ded4-4521-a540-1582d67b3e1a None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:47.052191 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:47.070154 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d7585-8bb2-4a0c-b232-22a758b59de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:47.077971 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d7585-8bb2-4a0c-b232-22a758b59de1 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:47.078366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d7585-8bb2-4a0c-b232-22a758b59de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:47.078776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d7585-8bb2-4a0c-b232-22a758b59de1 None admin] PATH_INFO: `/v3/groups/f9901716e50745a2ba3c9edb005aa761` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:47.080366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d7585-8bb2-4a0c-b232-22a758b59de1 None admin] RBAC: Authorizing `identity:get_group(group_id=f9901716e50745a2ba3c9edb005aa761)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:47.081149 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61947e09-88a7-450d-af50-da4e90f0a6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:47.081848 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61947e09-88a7-450d-af50-da4e90f0a6b2 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:47.082030 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d7585-8bb2-4a0c-b232-22a758b59de1 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:47.082182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61947e09-88a7-450d-af50-da4e90f0a6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:47.082460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61947e09-88a7-450d-af50-da4e90f0a6b2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:47.084165 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 215/415] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Aug 8 16:31:46 2022] GET /identity/v3/groups/f9901716e50745a2ba3c9edb005aa761 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:47.090999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d7585-8bb2-4a0c-b232-22a758b59de1 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:47.114399 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61947e09-88a7-450d-af50-da4e90f0a6b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:47.126648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-61947e09-88a7-450d-af50-da4e90f0a6b2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:47.126648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:47.126648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:47.126648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:47.126648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:47.126648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:47.126648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:47.126648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:47.126648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:47.126648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:47.126648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:47.126648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:47.126648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:47.126648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:47.126648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:47.126648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:47.128372 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:47.128372 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:47.128372 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:47.128372 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:47.128372 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:47.128372 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:47.128372 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:47.128372 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:47.128372 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:47.128372 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:47.128372 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:47.128372 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:47.128372 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:47.128372 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:47.128372 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:47.128372 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:47.131786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:47.131786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:47.131786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:47.131786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:47.131786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:47.131786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:47.131786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:47.131786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:47.131786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:47.131786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:47.131786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:47.131786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:47.131786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 201/416] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Aug 8 16:31:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:31:47.139099 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61947e09-88a7-450d-af50-da4e90f0a6b2 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:47.281200 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61947e09-88a7-450d-af50-da4e90f0a6b2 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:47.315143 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:47.327544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c963e9f2-e9a4-4706-80aa-26f598f57f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:47.328040 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963e9f2-e9a4-4706-80aa-26f598f57f81 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:47.328456 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963e9f2-e9a4-4706-80aa-26f598f57f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:47.328734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963e9f2-e9a4-4706-80aa-26f598f57f81 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:47.329680 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c963e9f2-e9a4-4706-80aa-26f598f57f81 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:47.330088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c963e9f2-e9a4-4706-80aa-26f598f57f81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:47.331688 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c963e9f2-e9a4-4706-80aa-26f598f57f81 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:47.344076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 202/417] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:31:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:47.352412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c963e9f2-e9a4-4706-80aa-26f598f57f81 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:47.374125 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d7585-8bb2-4a0c-b232-22a758b59de1 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:47.434412 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773fc9c-d07c-47e8-a228-6c7e04b6bb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:47.435081 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773fc9c-d07c-47e8-a228-6c7e04b6bb21 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:47.435354 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773fc9c-d07c-47e8-a228-6c7e04b6bb21 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:47.435654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773fc9c-d07c-47e8-a228-6c7e04b6bb21 None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:47.436818 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773fc9c-d07c-47e8-a228-6c7e04b6bb21 None admin] RBAC: Authorizing `identity:get_project(project_id=64dad222ddb64408b80bacb84185378f)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:47.438251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773fc9c-d07c-47e8-a228-6c7e04b6bb21 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:47.440243 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 216/418] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:47 2022] GET /identity/v3/projects/64dad222ddb64408b80bacb84185378f => generated 302 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:47.446198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773fc9c-d07c-47e8-a228-6c7e04b6bb21 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:47.543627 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:47.578591 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c963e9f2-e9a4-4706-80aa-26f598f57f81 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:47.590025 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773fc9c-d07c-47e8-a228-6c7e04b6bb21 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:47.629773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6a9e3-339d-400e-b132-2e5f77f29d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:47.630445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6a9e3-339d-400e-b132-2e5f77f29d49 None admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:47.630747 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6a9e3-339d-400e-b132-2e5f77f29d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:47.631016 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6a9e3-339d-400e-b132-2e5f77f29d49 None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f/groups/f9901716e50745a2ba3c9edb005aa761/roles/fd2599a3f408404599ecea94959efc4a` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:47.633214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6a9e3-339d-400e-b132-2e5f77f29d49 None admin] RBAC: Authorizing `identity:create_grant(project_id=64dad222ddb64408b80bacb84185378f, group_id=f9901716e50745a2ba3c9edb005aa761, role_id=fd2599a3f408404599ecea94959efc4a)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:47.635294 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6a9e3-339d-400e-b132-2e5f77f29d49 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:47.646598 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78641a95-e37a-4cc0-9237-80e69e55a811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:47.647309 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78641a95-e37a-4cc0-9237-80e69e55a811 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:47.647597 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78641a95-e37a-4cc0-9237-80e69e55a811 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:47.647881 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78641a95-e37a-4cc0-9237-80e69e55a811 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:47.711745 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 217/419] 10.222.0.13 () {58 vars in 1442 bytes} [Mon Aug 8 16:31:47 2022] PUT /identity/v3/projects/64dad222ddb64408b80bacb84185378f/groups/f9901716e50745a2ba3c9edb005aa761/roles/fd2599a3f408404599ecea94959efc4a => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:31:47.715556 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:47.738393 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78641a95-e37a-4cc0-9237-80e69e55a811 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:47.740541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78641a95-e37a-4cc0-9237-80e69e55a811 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:47.870655 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 203/420] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:47.876970 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6a9e3-339d-400e-b132-2e5f77f29d49 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:47.990761 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6a9e3-339d-400e-b132-2e5f77f29d49 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:48.010889 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:48.029320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f37abc-eb3c-4f5b-9049-03ec3a3d70d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:48.029961 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f37abc-eb3c-4f5b-9049-03ec3a3d70d0 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:48.030242 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f37abc-eb3c-4f5b-9049-03ec3a3d70d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:48.030510 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f37abc-eb3c-4f5b-9049-03ec3a3d70d0 None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:48.031707 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f37abc-eb3c-4f5b-9049-03ec3a3d70d0 None admin] RBAC: Authorizing `identity:get_project(project_id=88933471b063476fb90fe841b4e9b75b)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:48.033261 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f37abc-eb3c-4f5b-9049-03ec3a3d70d0 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:48.035279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 218/421] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:47 2022] GET /identity/v3/projects/88933471b063476fb90fe841b4e9b75b => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:48.041128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78641a95-e37a-4cc0-9237-80e69e55a811 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:48.144776 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:48.163107 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78641a95-e37a-4cc0-9237-80e69e55a811 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:48.204974 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba0b8d-6143-4aca-aaad-2b30254262f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:48.205610 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba0b8d-6143-4aca-aaad-2b30254262f5 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:48.205885 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba0b8d-6143-4aca-aaad-2b30254262f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:48.206149 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba0b8d-6143-4aca-aaad-2b30254262f5 None admin] PATH_INFO: `/v3/groups/f9901716e50745a2ba3c9edb005aa761` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:48.207296 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba0b8d-6143-4aca-aaad-2b30254262f5 None admin] RBAC: Authorizing `identity:get_group(group_id=f9901716e50745a2ba3c9edb005aa761)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:48.208812 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba0b8d-6143-4aca-aaad-2b30254262f5 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:48.214800 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 204/422] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Aug 8 16:31:48 2022] GET /identity/v3/groups/f9901716e50745a2ba3c9edb005aa761 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:48.216884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f37abc-eb3c-4f5b-9049-03ec3a3d70d0 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:48.231270 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3d633-a228-4d51-a179-f50c64883841 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:48.231616 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3d633-a228-4d51-a179-f50c64883841 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:48.231793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3d633-a228-4d51-a179-f50c64883841 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:48.236674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 205/423] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:48.239723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb73a3a-4d7d-42b9-8986-b207959f35d5 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:48.239998 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb73a3a-4d7d-42b9-8986-b207959f35d5 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:48.240264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb73a3a-4d7d-42b9-8986-b207959f35d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:48.263733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-3fb73a3a-4d7d-42b9-8986-b207959f35d5 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:48.264951 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-3fb73a3a-4d7d-42b9-8986-b207959f35d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:48.348791 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 206/424] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:48.383910 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb73a3a-4d7d-42b9-8986-b207959f35d5 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:48.406226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f37abc-eb3c-4f5b-9049-03ec3a3d70d0 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:48.456324 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdf0ec-bbd6-4e06-89f0-1d8f266a348b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:48.456947 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdf0ec-bbd6-4e06-89f0-1d8f266a348b None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:48.457224 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdf0ec-bbd6-4e06-89f0-1d8f266a348b None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:48.457522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdf0ec-bbd6-4e06-89f0-1d8f266a348b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:48.458223 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bdf0ec-bbd6-4e06-89f0-1d8f266a348b None admin] RBAC: Adding query filter params (group.id=f9901716e50745a2ba3c9edb005aa761, role.id=fd2599a3f408404599ecea94959efc4a, scope.project.id=88933471b063476fb90fe841b4e9b75b) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:48.458528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bdf0ec-bbd6-4e06-89f0-1d8f266a348b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:48.460219 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bdf0ec-bbd6-4e06-89f0-1d8f266a348b None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:48.482063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 219/425] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Aug 8 16:31:48 2022] GET /identity/v3/role_assignments?group.id=f9901716e50745a2ba3c9edb005aa761&scope.project.id=88933471b063476fb90fe841b4e9b75b&role.id=fd2599a3f408404599ecea94959efc4a => generated 611 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:48.511935 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:48.534135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb73a3a-4d7d-42b9-8986-b207959f35d5 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:48.574781 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748301c-d74e-455d-977c-7ed73c1ca14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:48.581540 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748301c-d74e-455d-977c-7ed73c1ca14b None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:48.582517 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748301c-d74e-455d-977c-7ed73c1ca14b None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:48.582968 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748301c-d74e-455d-977c-7ed73c1ca14b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:48.600578 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1748301c-d74e-455d-977c-7ed73c1ca14b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:48.606801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-1748301c-d74e-455d-977c-7ed73c1ca14b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:48.606801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:48.606801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:48.606801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:48.606801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:48.606801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:48.606801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:48.606801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:48.606801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:48.606801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:48.606801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:48.606801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:48.606801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:48.606801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:48.606801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:48.606801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:48.607831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:48.607831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:48.607831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:48.607831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:48.607831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:48.607831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:48.607831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:48.607831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:48.607831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:48.607831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:48.607831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:48.607831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:48.607831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:48.607831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:48.607831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:48.607831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:48.608656 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:48.608656 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:48.608656 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:48.608656 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:48.608656 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:48.608656 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:48.608656 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:48.608656 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:48.608656 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:48.608656 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:48.608656 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:48.608656 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:48.611838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 207/426] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:31:48.619824 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdf0ec-bbd6-4e06-89f0-1d8f266a348b None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:48.701442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:48.767644 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdf0ec-bbd6-4e06-89f0-1d8f266a348b None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:48.796207 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550d170-1ccb-4c02-8434-3e9769e166c7 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:48.796545 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550d170-1ccb-4c02-8434-3e9769e166c7 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:48.796746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550d170-1ccb-4c02-8434-3e9769e166c7 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:48.798266 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 208/427] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:48.804303 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63a198-1bf8-42c5-b01b-33308f933fc4 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:48.804891 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63a198-1bf8-42c5-b01b-33308f933fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:48.805427 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63a198-1bf8-42c5-b01b-33308f933fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:48.829545 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-2e63a198-1bf8-42c5-b01b-33308f933fc4 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:48.830853 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-2e63a198-1bf8-42c5-b01b-33308f933fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:48.832788 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d886f3-81fe-45ac-b570-1bc287ecaba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:48.833697 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d886f3-81fe-45ac-b570-1bc287ecaba0 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:48.834320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d886f3-81fe-45ac-b570-1bc287ecaba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:48.834896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d886f3-81fe-45ac-b570-1bc287ecaba0 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:48.835942 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d886f3-81fe-45ac-b570-1bc287ecaba0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:48.836477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d886f3-81fe-45ac-b570-1bc287ecaba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:48.839410 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d886f3-81fe-45ac-b570-1bc287ecaba0 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:48.851720 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 220/428] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:48.860160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d886f3-81fe-45ac-b570-1bc287ecaba0 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:48.938413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 209/429] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:31:48.969413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63a198-1bf8-42c5-b01b-33308f933fc4 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:49.016399 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d886f3-81fe-45ac-b570-1bc287ecaba0 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:49.031617 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:49.055089 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb6de0-8cd4-4d2c-8dfc-5c77e6772aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:49.056129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb6de0-8cd4-4d2c-8dfc-5c77e6772aad None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:49.056656 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb6de0-8cd4-4d2c-8dfc-5c77e6772aad None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:49.057336 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb6de0-8cd4-4d2c-8dfc-5c77e6772aad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:49.059612 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:49.098775 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63a198-1bf8-42c5-b01b-33308f933fc4 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:49.144907 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb6de0-8cd4-4d2c-8dfc-5c77e6772aad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:49.145373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb8d21-2e26-4f50-b395-4bf0ab019999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:49.146247 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb8d21-2e26-4f50-b395-4bf0ab019999 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:49.146388 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb6de0-8cd4-4d2c-8dfc-5c77e6772aad None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:49.146881 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb8d21-2e26-4f50-b395-4bf0ab019999 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:49.147365 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb8d21-2e26-4f50-b395-4bf0ab019999 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:49.159604 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb8d21-2e26-4f50-b395-4bf0ab019999 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:49.166363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-5fdb8d21-2e26-4f50-b395-4bf0ab019999 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:49.166363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:49.166363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:49.166363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:49.166363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:49.166363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:49.166363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:49.166363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:49.166363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:49.166363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:49.166363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:49.166363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:49.166363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:49.166363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:49.166363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:49.166363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:49.167313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:49.167313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:49.167313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:49.167313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:49.167313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:49.167313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:49.167313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:49.167313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:49.167313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:49.167313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:49.167313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:49.167313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:49.167313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:49.167313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:49.167313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:49.167313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:49.168121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:49.168121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:49.168121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:49.168121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:49.168121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:49.168121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:49.168121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:49.168121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:49.168121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:49.168121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:49.168121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:31:49.168121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:49.175030 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 210/430] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:49.177404 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb8d21-2e26-4f50-b395-4bf0ab019999 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:49.279182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 221/431] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:49.287123 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb6de0-8cd4-4d2c-8dfc-5c77e6772aad None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:49.305622 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:49.317814 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb8d21-2e26-4f50-b395-4bf0ab019999 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:49.355135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:49.357435 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e964e4-5177-496a-80c9-16bd6163ae83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:49.358043 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e964e4-5177-496a-80c9-16bd6163ae83 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:49.358319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e964e4-5177-496a-80c9-16bd6163ae83 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:49.358690 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e964e4-5177-496a-80c9-16bd6163ae83 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:49.359372 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e964e4-5177-496a-80c9-16bd6163ae83 None admin] RBAC: Adding query filter params (name=member) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:49.359669 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e964e4-5177-496a-80c9-16bd6163ae83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:49.361061 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e964e4-5177-496a-80c9-16bd6163ae83 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:49.370968 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 211/432] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:31:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:49.372235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930b963-0479-4655-afe9-03640c5b7331 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:49.372513 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930b963-0479-4655-afe9-03640c5b7331 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:49.372772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930b963-0479-4655-afe9-03640c5b7331 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:49.374607 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 212/433] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:49.376616 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930b963-0479-4655-afe9-03640c5b7331 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:49.419164 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb6de0-8cd4-4d2c-8dfc-5c77e6772aad None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:49.470110 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482b525-3330-4dbb-a0e2-0493b4521baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:49.470788 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482b525-3330-4dbb-a0e2-0493b4521baf None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:49.471493 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482b525-3330-4dbb-a0e2-0493b4521baf None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:49.471771 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482b525-3330-4dbb-a0e2-0493b4521baf None admin] PATH_INFO: `/v3/projects/88933471b063476fb90fe841b4e9b75b` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:49.473202 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482b525-3330-4dbb-a0e2-0493b4521baf None admin] RBAC: Authorizing `identity:get_project(project_id=88933471b063476fb90fe841b4e9b75b)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:49.475496 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482b525-3330-4dbb-a0e2-0493b4521baf None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:49.483267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 222/434] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:49 2022] GET /identity/v3/projects/88933471b063476fb90fe841b4e9b75b => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:49.483267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bc58c-72a1-4bc8-9bb7-044c06bfe4d4 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:49.483267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bc58c-72a1-4bc8-9bb7-044c06bfe4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:49.483267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bc58c-72a1-4bc8-9bb7-044c06bfe4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:49.496610 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930b963-0479-4655-afe9-03640c5b7331 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:49.502421 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-530bc58c-72a1-4bc8-9bb7-044c06bfe4d4 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:49.503648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-530bc58c-72a1-4bc8-9bb7-044c06bfe4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:49.536609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbce7b1-d6de-4cf9-8fa9-5cd6bf1626a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:49.537250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbce7b1-d6de-4cf9-8fa9-5cd6bf1626a7 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:49.537451 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbce7b1-d6de-4cf9-8fa9-5cd6bf1626a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:49.537720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbce7b1-d6de-4cf9-8fa9-5cd6bf1626a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:49.579156 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 223/435] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:49.580231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530bc58c-72a1-4bc8-9bb7-044c06bfe4d4 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:49.605067 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:49.610556 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:49.618699 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbce7b1-d6de-4cf9-8fa9-5cd6bf1626a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:49.620972 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbce7b1-d6de-4cf9-8fa9-5cd6bf1626a7 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:49.703115 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530bc58c-72a1-4bc8-9bb7-044c06bfe4d4 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:49.733638 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 213/436] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:49.733638 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbce7b1-d6de-4cf9-8fa9-5cd6bf1626a7 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:49.751092 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2445f-53cc-4296-983c-53fb6f0f0f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:49.751767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2445f-53cc-4296-983c-53fb6f0f0f36 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:49.752043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2445f-53cc-4296-983c-53fb6f0f0f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:49.752307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2445f-53cc-4296-983c-53fb6f0f0f36 None admin] PATH_INFO: `/v3/groups/f9901716e50745a2ba3c9edb005aa761` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:49.753478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2445f-53cc-4296-983c-53fb6f0f0f36 None admin] RBAC: Authorizing `identity:get_group(group_id=f9901716e50745a2ba3c9edb005aa761)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:49.755020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2445f-53cc-4296-983c-53fb6f0f0f36 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:49.756814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 224/437] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Aug 8 16:31:49 2022] GET /identity/v3/groups/f9901716e50745a2ba3c9edb005aa761 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:49.762916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2445f-53cc-4296-983c-53fb6f0f0f36 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:49.832695 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:49.841450 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbce7b1-d6de-4cf9-8fa9-5cd6bf1626a7 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:49.850934 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:49.876033 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2445f-53cc-4296-983c-53fb6f0f0f36 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:49.878686 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc3037-8f23-4a8a-b254-9f3178921de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:49.879400 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc3037-8f23-4a8a-b254-9f3178921de4 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:49.879682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc3037-8f23-4a8a-b254-9f3178921de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:49.879955 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc3037-8f23-4a8a-b254-9f3178921de4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:49.891153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc3037-8f23-4a8a-b254-9f3178921de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:49.896103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-19fc3037-8f23-4a8a-b254-9f3178921de4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:31:49.896103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:49.896103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:49.896103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:49.896103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:49.896103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:49.896103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:49.896103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:49.896103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:49.896103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:49.896103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:49.896103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:49.896103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:49.896103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:49.896103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:49.896103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:49.897034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:49.897034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:49.897034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:49.897034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:49.897034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:49.897034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:49.897034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:49.897034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:49.897034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:49.897034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:49.897034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:49.897034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:49.897034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:49.897034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:49.897034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:49.897034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:49.897860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:49.897860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:49.897860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:49.897860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:49.897860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:49.897860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:49.897860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:49.897860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:49.897860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:49.897860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:49.897860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:31:49.897860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:49.902764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 214/438] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:31:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:49.903532 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc3037-8f23-4a8a-b254-9f3178921de4 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:49.913970 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef3475-981e-4dc3-b087-8acca9a9ac70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:49.914661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef3475-981e-4dc3-b087-8acca9a9ac70 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:49.914935 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef3475-981e-4dc3-b087-8acca9a9ac70 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:49.915198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef3475-981e-4dc3-b087-8acca9a9ac70 None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:49.917185 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef3475-981e-4dc3-b087-8acca9a9ac70 None admin] RBAC: Authorizing `identity:get_project(project_id=64dad222ddb64408b80bacb84185378f)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:49.918776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef3475-981e-4dc3-b087-8acca9a9ac70 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:49.922316 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 225/439] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:49 2022] GET /identity/v3/projects/64dad222ddb64408b80bacb84185378f => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:49.922316 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef3475-981e-4dc3-b087-8acca9a9ac70 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:50.035968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef3475-981e-4dc3-b087-8acca9a9ac70 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:50.038782 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc3037-8f23-4a8a-b254-9f3178921de4 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:50.077352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44732ab-84ed-47a8-ace9-57a626f93a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:50.078130 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44732ab-84ed-47a8-ace9-57a626f93a4b None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:50.078407 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44732ab-84ed-47a8-ace9-57a626f93a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:50.078706 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44732ab-84ed-47a8-ace9-57a626f93a4b None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:50.079363 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44732ab-84ed-47a8-ace9-57a626f93a4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:50.079655 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44732ab-84ed-47a8-ace9-57a626f93a4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:50.081005 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44732ab-84ed-47a8-ace9-57a626f93a4b None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:50.083132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2182f-3975-40a2-b8a5-850aaa32d664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:50.083970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2182f-3975-40a2-b8a5-850aaa32d664 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:50.084383 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2182f-3975-40a2-b8a5-850aaa32d664 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:50.084822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2182f-3975-40a2-b8a5-850aaa32d664 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:50.085691 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2182f-3975-40a2-b8a5-850aaa32d664 None admin] RBAC: Adding query filter params (group.id=f9901716e50745a2ba3c9edb005aa761, role.id=04bf40fcc26347de8167fc215f62c7aa, scope.project.id=88933471b063476fb90fe841b4e9b75b) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:50.086132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2182f-3975-40a2-b8a5-850aaa32d664 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:50.088470 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2182f-3975-40a2-b8a5-850aaa32d664 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:50.090869 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 226/440] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:31:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:50.091857 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44732ab-84ed-47a8-ace9-57a626f93a4b None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:50.099858 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 215/441] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Aug 8 16:31:49 2022] GET /identity/v3/role_assignments?group.id=f9901716e50745a2ba3c9edb005aa761&scope.project.id=88933471b063476fb90fe841b4e9b75b&role.id=04bf40fcc26347de8167fc215f62c7aa => generated 611 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:50.100846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2182f-3975-40a2-b8a5-850aaa32d664 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:50.108051 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:50.124506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:50.223390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44732ab-84ed-47a8-ace9-57a626f93a4b None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:50.244899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2182f-3975-40a2-b8a5-850aaa32d664 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:50.265473 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f833d-89f0-4880-93fb-b8f5ca056339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:50.266162 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f833d-89f0-4880-93fb-b8f5ca056339 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:50.266376 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f833d-89f0-4880-93fb-b8f5ca056339 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:50.266668 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f833d-89f0-4880-93fb-b8f5ca056339 None admin] PATH_INFO: `/v3/groups/f9901716e50745a2ba3c9edb005aa761` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:50.267964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f833d-89f0-4880-93fb-b8f5ca056339 None admin] RBAC: Authorizing `identity:get_group(group_id=f9901716e50745a2ba3c9edb005aa761)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:50.269266 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f833d-89f0-4880-93fb-b8f5ca056339 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:50.271335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 227/442] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Aug 8 16:31:50 2022] GET /identity/v3/groups/f9901716e50745a2ba3c9edb005aa761 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:50.278189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f833d-89f0-4880-93fb-b8f5ca056339 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:50.296964 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc18ea2-11c7-41ed-933e-edecbf304792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:50.297934 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc18ea2-11c7-41ed-933e-edecbf304792 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:50.298358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc18ea2-11c7-41ed-933e-edecbf304792 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:50.298862 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc18ea2-11c7-41ed-933e-edecbf304792 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:50.381603 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:50.389089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc18ea2-11c7-41ed-933e-edecbf304792 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:50.390631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc18ea2-11c7-41ed-933e-edecbf304792 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:50.411361 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:50.411550 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f833d-89f0-4880-93fb-b8f5ca056339 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:50.458421 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811eb420-a560-4001-a0e1-b5ad0b627a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:50.459072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811eb420-a560-4001-a0e1-b5ad0b627a18 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:50.459351 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811eb420-a560-4001-a0e1-b5ad0b627a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:50.459626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811eb420-a560-4001-a0e1-b5ad0b627a18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:50.460495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811eb420-a560-4001-a0e1-b5ad0b627a18 None admin] RBAC: Adding query filter params (group.id=f9901716e50745a2ba3c9edb005aa761, role.id=04bf40fcc26347de8167fc215f62c7aa, scope.project.id=64dad222ddb64408b80bacb84185378f) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:50.460886 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811eb420-a560-4001-a0e1-b5ad0b627a18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:50.462475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811eb420-a560-4001-a0e1-b5ad0b627a18 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:50.470056 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 228/443] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Aug 8 16:31:50 2022] GET /identity/v3/role_assignments?group.id=f9901716e50745a2ba3c9edb005aa761&scope.project.id=64dad222ddb64408b80bacb84185378f&role.id=04bf40fcc26347de8167fc215f62c7aa => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:50.471342 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05395f8-9169-4161-8c60-ababa98b27e9 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:50.471619 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05395f8-9169-4161-8c60-ababa98b27e9 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:50.471887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05395f8-9169-4161-8c60-ababa98b27e9 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:50.473477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 229/444] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:50.479934 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c9dcb-d908-43f4-9aed-7c0df56f0eb8 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:50.480251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c9dcb-d908-43f4-9aed-7c0df56f0eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:50.480481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c9dcb-d908-43f4-9aed-7c0df56f0eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:50.504164 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-b53c9dcb-d908-43f4-9aed-7c0df56f0eb8 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:50.505150 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b53c9dcb-d908-43f4-9aed-7c0df56f0eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:50.524018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 216/445] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:50.531232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc18ea2-11c7-41ed-933e-edecbf304792 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:50.615248 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 230/446] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:31:50.637178 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c9dcb-d908-43f4-9aed-7c0df56f0eb8 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:50.687980 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc18ea2-11c7-41ed-933e-edecbf304792 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:50.712168 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:50.722989 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:50.742487 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918efd06-a543-4e99-8a3a-6054a3c72a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:50.744704 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918efd06-a543-4e99-8a3a-6054a3c72a9b None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:50.745172 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918efd06-a543-4e99-8a3a-6054a3c72a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:50.746035 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918efd06-a543-4e99-8a3a-6054a3c72a9b None admin] PATH_INFO: `/v3/projects/3a8d1eb28476411285e5ac5c0ab9ab53` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:50.747729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918efd06-a543-4e99-8a3a-6054a3c72a9b None admin] RBAC: Authorizing `identity:get_project(project_id=3a8d1eb28476411285e5ac5c0ab9ab53)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:50.749281 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918efd06-a543-4e99-8a3a-6054a3c72a9b None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:50.750938 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 217/447] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:50 2022] GET /identity/v3/projects/3a8d1eb28476411285e5ac5c0ab9ab53 => generated 344 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:50.756856 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918efd06-a543-4e99-8a3a-6054a3c72a9b None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:50.760831 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c9dcb-d908-43f4-9aed-7c0df56f0eb8 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:50.802617 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b36e0-5a6e-400d-8552-a3f655afeefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:50.803255 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b36e0-5a6e-400d-8552-a3f655afeefd None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:50.803528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b36e0-5a6e-400d-8552-a3f655afeefd None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:50.803793 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b36e0-5a6e-400d-8552-a3f655afeefd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:50.816535 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b36e0-5a6e-400d-8552-a3f655afeefd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:50.821198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-0e3b36e0-5a6e-400d-8552-a3f655afeefd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:50.821198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:50.821198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:50.821198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:50.821198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:50.821198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:50.821198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:50.821198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:50.821198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:50.821198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:50.821198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:50.821198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:50.821198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:50.821198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:50.821198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:50.821198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:50.822877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:50.822877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:50.822877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:50.822877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:50.822877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:50.822877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:50.822877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:50.822877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:50.822877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:50.822877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:50.822877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:50.822877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:50.822877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:50.822877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:50.822877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:50.822877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:50.823674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:50.823674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:50.823674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:50.823674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:50.823674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:50.823674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:50.823674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:50.823674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:50.823674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:50.823674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:50.823674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:50.823674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:50.823674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 231/448] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Aug 8 16:31:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:50.829026 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b36e0-5a6e-400d-8552-a3f655afeefd None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:50.891807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918efd06-a543-4e99-8a3a-6054a3c72a9b None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:50.936648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a9dd4-138b-4299-937e-853b9fa16a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:50.937291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a9dd4-138b-4299-937e-853b9fa16a93 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:50.937568 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a9dd4-138b-4299-937e-853b9fa16a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:50.937837 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a9dd4-138b-4299-937e-853b9fa16a93 None admin] PATH_INFO: `/v3/groups/8f09983cdd674c14abd3004382998b60` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:50.938994 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a9dd4-138b-4299-937e-853b9fa16a93 None admin] RBAC: Authorizing `identity:get_group(group_id=8f09983cdd674c14abd3004382998b60)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:50.940457 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a9dd4-138b-4299-937e-853b9fa16a93 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:50.942183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 218/449] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Aug 8 16:31:50 2022] GET /identity/v3/groups/8f09983cdd674c14abd3004382998b60 => generated 229 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:50.948616 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a9dd4-138b-4299-937e-853b9fa16a93 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:50.952530 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b36e0-5a6e-400d-8552-a3f655afeefd None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:50.981946 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:50.988836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df05ffe-03dd-497f-81ab-ee44e75a36c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:50.989869 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df05ffe-03dd-497f-81ab-ee44e75a36c8 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:50.990149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df05ffe-03dd-497f-81ab-ee44e75a36c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:50.990419 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df05ffe-03dd-497f-81ab-ee44e75a36c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:50.991486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df05ffe-03dd-497f-81ab-ee44e75a36c8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:50.991802 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df05ffe-03dd-497f-81ab-ee44e75a36c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:50.993497 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df05ffe-03dd-497f-81ab-ee44e75a36c8 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:51.002206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 232/450] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:31:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:51.006944 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:51.007987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df05ffe-03dd-497f-81ab-ee44e75a36c8 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:51.050339 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a9dd4-138b-4299-937e-853b9fa16a93 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:51.084469 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff8a35-32b9-439d-8db9-fb13f813189a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:51.085077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff8a35-32b9-439d-8db9-fb13f813189a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:51.085355 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff8a35-32b9-439d-8db9-fb13f813189a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:51.085623 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff8a35-32b9-439d-8db9-fb13f813189a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:51.086350 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff8a35-32b9-439d-8db9-fb13f813189a None admin] RBAC: Adding query filter params (group.id=8f09983cdd674c14abd3004382998b60, role.id=e578ef831127463d9b48cc89fe20329c, scope.project.id=3a8d1eb28476411285e5ac5c0ab9ab53) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:51.086689 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff8a35-32b9-439d-8db9-fb13f813189a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:51.088815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff8a35-32b9-439d-8db9-fb13f813189a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:51.094945 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 219/451] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Aug 8 16:31:50 2022] GET /identity/v3/role_assignments?group.id=8f09983cdd674c14abd3004382998b60&scope.project.id=3a8d1eb28476411285e5ac5c0ab9ab53&role.id=e578ef831127463d9b48cc89fe20329c => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:51.108045 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df05ffe-03dd-497f-81ab-ee44e75a36c8 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:51.146246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b7045-f9e1-4d06-855e-7a7393363ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:51.146884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b7045-f9e1-4d06-855e-7a7393363ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:51.147099 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b7045-f9e1-4d06-855e-7a7393363ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:51.147364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b7045-f9e1-4d06-855e-7a7393363ff0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:51.210648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:51.221924 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b7045-f9e1-4d06-855e-7a7393363ff0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:51.223433 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b7045-f9e1-4d06-855e-7a7393363ff0 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:51.327486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 233/452] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:31:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:31:51.333839 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff8a35-32b9-439d-8db9-fb13f813189a None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:51.434838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff8a35-32b9-439d-8db9-fb13f813189a None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:51.452270 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:51.468595 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59d95e-5078-45de-bf38-f3bd1980b5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:51.469209 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59d95e-5078-45de-bf38-f3bd1980b5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:51.469482 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59d95e-5078-45de-bf38-f3bd1980b5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:51.469742 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59d95e-5078-45de-bf38-f3bd1980b5e6 None admin] PATH_INFO: `/v3/projects/64dad222ddb64408b80bacb84185378f` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:51.470891 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e59d95e-5078-45de-bf38-f3bd1980b5e6 None admin] RBAC: Authorizing `identity:get_project(project_id=64dad222ddb64408b80bacb84185378f)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:51.475285 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e59d95e-5078-45de-bf38-f3bd1980b5e6 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:51.475285 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 220/453] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:31:51 2022] GET /identity/v3/projects/64dad222ddb64408b80bacb84185378f => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:51.479680 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b7045-f9e1-4d06-855e-7a7393363ff0 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:51.568482 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:51.577656 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b7045-f9e1-4d06-855e-7a7393363ff0 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:51.612882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c7b60-5c84-4a7a-8711-29a1ed11290b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:51.613505 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c7b60-5c84-4a7a-8711-29a1ed11290b None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:51.613788 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c7b60-5c84-4a7a-8711-29a1ed11290b None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:51.614055 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c7b60-5c84-4a7a-8711-29a1ed11290b None admin] PATH_INFO: `/v3/groups/f9901716e50745a2ba3c9edb005aa761` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:51.615161 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c7b60-5c84-4a7a-8711-29a1ed11290b None admin] RBAC: Authorizing `identity:get_group(group_id=f9901716e50745a2ba3c9edb005aa761)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:51.616662 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c7b60-5c84-4a7a-8711-29a1ed11290b None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:51.618225 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 234/454] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Aug 8 16:31:51 2022] GET /identity/v3/groups/f9901716e50745a2ba3c9edb005aa761 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:31:51.624121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59d95e-5078-45de-bf38-f3bd1980b5e6 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:51.721161 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59d95e-5078-45de-bf38-f3bd1980b5e6 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:51.754052 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b3796-1747-4a0d-9125-03b398fe688d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:51.754663 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b3796-1747-4a0d-9125-03b398fe688d None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:51.754936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b3796-1747-4a0d-9125-03b398fe688d None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:51.755201 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b3796-1747-4a0d-9125-03b398fe688d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:51.755910 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b3796-1747-4a0d-9125-03b398fe688d None admin] RBAC: Adding query filter params (group.id=f9901716e50745a2ba3c9edb005aa761, role.id=fd2599a3f408404599ecea94959efc4a, scope.project.id=64dad222ddb64408b80bacb84185378f) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:51.756210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b3796-1747-4a0d-9125-03b398fe688d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:51.757626 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b3796-1747-4a0d-9125-03b398fe688d None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:51.763430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 221/455] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Aug 8 16:31:51 2022] GET /identity/v3/role_assignments?group.id=f9901716e50745a2ba3c9edb005aa761&scope.project.id=64dad222ddb64408b80bacb84185378f&role.id=fd2599a3f408404599ecea94959efc4a => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:54.896433 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff4c9b-748f-4140-90c8-a9e2be4e524a None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:54.897491 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff4c9b-748f-4140-90c8-a9e2be4e524a None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:54.897974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff4c9b-748f-4140-90c8-a9e2be4e524a None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:54.899889 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 235/456] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:54.906637 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4051a-a80f-4886-8ed0-2bff6383a520 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:54.907738 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4051a-a80f-4886-8ed0-2bff6383a520 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:54.908358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4051a-a80f-4886-8ed0-2bff6383a520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:54.949595 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-e6d4051a-a80f-4886-8ed0-2bff6383a520 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:54.951440 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-e6d4051a-a80f-4886-8ed0-2bff6383a520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:55.062902 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:55.110417 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 222/457] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:55.148920 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff4c9b-748f-4140-90c8-a9e2be4e524a None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:55.187724 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61b6a3-9797-4a64-8521-e72a649be850 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:55.187724 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61b6a3-9797-4a64-8521-e72a649be850 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:55.187724 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61b6a3-9797-4a64-8521-e72a649be850 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:55.188368 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 223/458] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:31:55.193759 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e32a1e-5ada-424b-a2f4-d0e07b311e2f None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:55.194220 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e32a1e-5ada-424b-a2f4-d0e07b311e2f None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:55.194652 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e32a1e-5ada-424b-a2f4-d0e07b311e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:55.219101 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-55e32a1e-5ada-424b-a2f4-d0e07b311e2f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:55.220660 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-55e32a1e-5ada-424b-a2f4-d0e07b311e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:55.299379 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff4c9b-748f-4140-90c8-a9e2be4e524a None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:55.310709 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 224/459] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:31:55.312294 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9969e-bdae-4990-8696-ad172b660977 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:55.312674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9969e-bdae-4990-8696-ad172b660977 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:55.313082 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9969e-bdae-4990-8696-ad172b660977 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:55.314470 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 225/460] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:55.319720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cba834-122b-4377-a81d-629cda47561c None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:55.320245 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cba834-122b-4377-a81d-629cda47561c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:55.320709 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cba834-122b-4377-a81d-629cda47561c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:55.338051 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a2898-f29d-49a2-a7dc-6c0b65b065e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:55.339016 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a2898-f29d-49a2-a7dc-6c0b65b065e8 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:55.339448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a2898-f29d-49a2-a7dc-6c0b65b065e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:55.339867 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a2898-f29d-49a2-a7dc-6c0b65b065e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:55.351047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a2898-f29d-49a2-a7dc-6c0b65b065e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:55.352505 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a2898-f29d-49a2-a7dc-6c0b65b065e8 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:55.353557 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-c4cba834-122b-4377-a81d-629cda47561c None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:55.355115 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-c4cba834-122b-4377-a81d-629cda47561c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:55.369678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-153a2898-f29d-49a2-a7dc-6c0b65b065e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:31:55.369678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:55.369678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:55.369678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:55.369678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:55.369678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:55.369678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:55.369678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:55.369678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:55.369678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:55.369678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:55.369678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:55.369678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:31:55.369678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:31:55.369678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:31:55.369678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:31:55.371334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:55.371334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:55.371334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:55.371334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:55.371334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:55.371334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:55.371334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:55.371334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:55.371334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:55.371334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:55.371334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:55.371334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:55.371334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:55.371334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:55.371334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:55.371334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:55.371334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:31:55.372193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:31:55.372193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:31:55.372193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:55.372193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 236/461] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:31:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:55.372425 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928c434-7273-4134-b732-ebf45bcf0491 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:55.372742 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928c434-7273-4134-b732-ebf45bcf0491 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:55.372838 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928c434-7273-4134-b732-ebf45bcf0491 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:55.377834 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 237/462] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:55.377834 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928c434-7273-4134-b732-ebf45bcf0491 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:55.391674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:55.438981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 226/463] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:31:55.446665 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc2a71-3b8c-48c7-a28f-7171b445e856 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:55.446665 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc2a71-3b8c-48c7-a28f-7171b445e856 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:55.446665 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc2a71-3b8c-48c7-a28f-7171b445e856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:55.466984 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-18fc2a71-3b8c-48c7-a28f-7171b445e856 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:55.468877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-18fc2a71-3b8c-48c7-a28f-7171b445e856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:55.497536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928c434-7273-4134-b732-ebf45bcf0491 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:55.504832 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:55.533909 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50f209-d732-42f7-835d-23dd151e6cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:55.534541 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50f209-d732-42f7-835d-23dd151e6cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:55.534844 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50f209-d732-42f7-835d-23dd151e6cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:55.535097 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50f209-d732-42f7-835d-23dd151e6cb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:55.540077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 227/464] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:55.540776 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fc2a71-3b8c-48c7-a28f-7171b445e856 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:55.551479 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50f209-d732-42f7-835d-23dd151e6cb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:55.553904 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50f209-d732-42f7-835d-23dd151e6cb6 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:55.572366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-5d50f209-d732-42f7-835d-23dd151e6cb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:31:55.572366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:55.572366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:55.572366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:55.572366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:55.572366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:55.572366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:55.572366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:55.572366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:55.572366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:55.572366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:55.572366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:55.572366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:31:55.572366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:31:55.572366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:31:55.572366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:31:55.573477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:55.573477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:55.573477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:55.573477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:55.573477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:55.573477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:55.573477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:55.573477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:55.573477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:55.573477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:55.573477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:55.573477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:55.573477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:55.573477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:55.573477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:55.573477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:55.573477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:31:55.574342 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:31:55.574342 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:31:55.574342 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:55.574342 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 238/465] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:31:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:55.575014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50f209-d732-42f7-835d-23dd151e6cb6 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:55.642686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:55.655104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fc2a71-3b8c-48c7-a28f-7171b445e856 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:55.687045 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50f209-d732-42f7-835d-23dd151e6cb6 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:55.696798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff1988-b286-42d1-91e2-085ed063117e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:55.697743 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff1988-b286-42d1-91e2-085ed063117e None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:55.698684 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff1988-b286-42d1-91e2-085ed063117e None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:55.699142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff1988-b286-42d1-91e2-085ed063117e None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:55.700276 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff1988-b286-42d1-91e2-085ed063117e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:55.700762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff1988-b286-42d1-91e2-085ed063117e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:55.703032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff1988-b286-42d1-91e2-085ed063117e None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:55.719188 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 228/466] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:31:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:55.720442 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff1988-b286-42d1-91e2-085ed063117e None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:55.726526 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60aa325-0242-4c42-b9ff-8d906793cbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:55.727438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60aa325-0242-4c42-b9ff-8d906793cbe9 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:55.727719 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60aa325-0242-4c42-b9ff-8d906793cbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:55.728501 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60aa325-0242-4c42-b9ff-8d906793cbe9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:55.742439 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60aa325-0242-4c42-b9ff-8d906793cbe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:55.745394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60aa325-0242-4c42-b9ff-8d906793cbe9 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:55.756253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:55.762843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-c60aa325-0242-4c42-b9ff-8d906793cbe9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:31:55.762843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:55.762843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:55.762843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:55.762843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:55.762843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:55.762843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:55.762843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:55.762843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:55.762843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:55.762843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:55.762843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:55.762843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:31:55.762843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:31:55.762843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:31:55.762843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:31:55.769481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:55.769481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:55.769481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:55.769481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:55.769481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:55.769481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:55.769481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:55.769481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:55.769481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:55.769481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:55.769481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:55.769481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:55.769481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:55.769481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:55.769481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:55.769481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:55.769481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:31:55.770545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:31:55.770545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:31:55.770545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:55.770545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 239/467] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:31:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:55.770545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60aa325-0242-4c42-b9ff-8d906793cbe9 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:55.836001 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff1988-b286-42d1-91e2-085ed063117e None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:55.878169 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acb05d-5b52-47be-a64c-13d23334ac0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:55.878936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acb05d-5b52-47be-a64c-13d23334ac0d None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:55.879285 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acb05d-5b52-47be-a64c-13d23334ac0d None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:55.879791 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acb05d-5b52-47be-a64c-13d23334ac0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:55.883208 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60aa325-0242-4c42-b9ff-8d906793cbe9 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:55.890544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57acb05d-5b52-47be-a64c-13d23334ac0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:55.892042 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57acb05d-5b52-47be-a64c-13d23334ac0d None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:55.902687 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:55.909053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-57acb05d-5b52-47be-a64c-13d23334ac0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:31:55.909053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:55.909053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:55.909053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:55.909053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:55.909053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:55.909053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:55.909053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:55.909053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:55.909053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:55.909053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:55.909053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:55.909053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:31:55.909053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:31:55.909053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:31:55.909053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:31:55.910762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:55.910762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:55.910762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:55.910762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:55.910762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:55.910762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:55.910762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:55.910762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:55.910762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:55.910762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:55.910762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:55.910762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:55.910762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:55.910762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:55.910762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:55.910762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:55.910762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:31:55.916847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:31:55.916847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:31:55.916847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:55.916847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 229/468] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:31:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:55.916847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acb05d-5b52-47be-a64c-13d23334ac0d None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:55.935743 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71770891-8933-411e-a243-56f430d35514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:55.936631 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71770891-8933-411e-a243-56f430d35514 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:55.937427 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71770891-8933-411e-a243-56f430d35514 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:55.937704 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71770891-8933-411e-a243-56f430d35514 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:55.938668 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71770891-8933-411e-a243-56f430d35514 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:55.938973 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71770891-8933-411e-a243-56f430d35514 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:55.940981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71770891-8933-411e-a243-56f430d35514 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:55.963122 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 240/469] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:31:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:55.963122 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71770891-8933-411e-a243-56f430d35514 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:56.015812 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:56.042008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acb05d-5b52-47be-a64c-13d23334ac0d None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:56.083348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebebf20-a42a-4da4-a41c-e34e39978216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:56.084163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71770891-8933-411e-a243-56f430d35514 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:56.084539 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebebf20-a42a-4da4-a41c-e34e39978216 None admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:56.085011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebebf20-a42a-4da4-a41c-e34e39978216 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:56.085490 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebebf20-a42a-4da4-a41c-e34e39978216 None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:56.086303 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebebf20-a42a-4da4-a41c-e34e39978216 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:56.088975 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebebf20-a42a-4da4-a41c-e34e39978216 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:56.121720 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1802f07-771e-4186-8531-426bb3e700a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:56.122327 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1802f07-771e-4186-8531-426bb3e700a9 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:56.122642 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1802f07-771e-4186-8531-426bb3e700a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:56.122920 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1802f07-771e-4186-8531-426bb3e700a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:56.123556 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1802f07-771e-4186-8531-426bb3e700a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:56.123854 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1802f07-771e-4186-8531-426bb3e700a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:56.125284 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1802f07-771e-4186-8531-426bb3e700a9 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:56.136595 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 241/470] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:31:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:56.136595 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1802f07-771e-4186-8531-426bb3e700a9 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:56.175504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:56.185273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 230/471] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Aug 8 16:31:55 2022] POST /identity/v3/users => generated 246 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:31:56.186162 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebebf20-a42a-4da4-a41c-e34e39978216 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:56.250105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1802f07-771e-4186-8531-426bb3e700a9 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:56.293379 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c56e55f-3715-45de-8400-fc532c5e1d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:56.294509 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56e55f-3715-45de-8400-fc532c5e1d8b None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:56.295050 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56e55f-3715-45de-8400-fc532c5e1d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:56.295554 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56e55f-3715-45de-8400-fc532c5e1d8b None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:56.296417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c56e55f-3715-45de-8400-fc532c5e1d8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:56.296985 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c56e55f-3715-45de-8400-fc532c5e1d8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:56.298857 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c56e55f-3715-45de-8400-fc532c5e1d8b None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:56.306306 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebebf20-a42a-4da4-a41c-e34e39978216 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:56.313599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 242/472] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:56.314892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c56e55f-3715-45de-8400-fc532c5e1d8b None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:56.352440 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441add7d-c3b6-4919-9efa-a4ee3ff6530b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:56.353699 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441add7d-c3b6-4919-9efa-a4ee3ff6530b None admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:56.353699 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441add7d-c3b6-4919-9efa-a4ee3ff6530b None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:56.353699 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441add7d-c3b6-4919-9efa-a4ee3ff6530b None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:56.354876 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441add7d-c3b6-4919-9efa-a4ee3ff6530b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:56.355890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441add7d-c3b6-4919-9efa-a4ee3ff6530b None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:56.366122 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:56.421311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:56.427582 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c56e55f-3715-45de-8400-fc532c5e1d8b None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:56.461692 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 231/473] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Aug 8 16:31:56 2022] POST /identity/v3/users => generated 248 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:31:56.461692 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c38609-ce15-4b54-bff5-1c6593252de6 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:56.461692 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c38609-ce15-4b54-bff5-1c6593252de6 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:56.461692 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c38609-ce15-4b54-bff5-1c6593252de6 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:56.461692 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 232/474] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:56.461692 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c38609-ce15-4b54-bff5-1c6593252de6 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:56.471315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc40654-619d-4dca-808a-d93274f6e88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:56.471951 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc40654-619d-4dca-808a-d93274f6e88b None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:56.472222 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc40654-619d-4dca-808a-d93274f6e88b None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:56.472488 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc40654-619d-4dca-808a-d93274f6e88b None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:56.473263 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc40654-619d-4dca-808a-d93274f6e88b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:56.474781 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc40654-619d-4dca-808a-d93274f6e88b None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:56.569131 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 243/475] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Aug 8 16:31:56 2022] POST /identity/v3/users => generated 248 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:31:56.570314 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8a770-dab5-4e40-a131-ea48c26fa359 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:56.570605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8a770-dab5-4e40-a131-ea48c26fa359 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:56.570879 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8a770-dab5-4e40-a131-ea48c26fa359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:56.613688 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-6ec8a770-dab5-4e40-a131-ea48c26fa359 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:56.614976 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-6ec8a770-dab5-4e40-a131-ea48c26fa359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:56.620971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c38609-ce15-4b54-bff5-1c6593252de6 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:56.679579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad4ae6-0625-4f18-b1a2-43da50b920b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:56.680207 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad4ae6-0625-4f18-b1a2-43da50b920b3 None admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:56.680485 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad4ae6-0625-4f18-b1a2-43da50b920b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:56.680806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad4ae6-0625-4f18-b1a2-43da50b920b3 None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:56.682459 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad4ae6-0625-4f18-b1a2-43da50b920b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:56.684059 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad4ae6-0625-4f18-b1a2-43da50b920b3 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:56.716505 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 244/476] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:56.740208 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8a770-dab5-4e40-a131-ea48c26fa359 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:56.781360 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 233/477] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Aug 8 16:31:56 2022] POST /identity/v3/users => generated 249 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:31:56.842097 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:56.865253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8a770-dab5-4e40-a131-ea48c26fa359 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:56.905392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868279ab-6714-444e-bdaa-bc7fef8d3c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:56.906032 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868279ab-6714-444e-bdaa-bc7fef8d3c33 None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:56.906296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868279ab-6714-444e-bdaa-bc7fef8d3c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:56.906597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868279ab-6714-444e-bdaa-bc7fef8d3c33 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:56.907303 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868279ab-6714-444e-bdaa-bc7fef8d3c33 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:56.908859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868279ab-6714-444e-bdaa-bc7fef8d3c33 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:56.946036 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.sql.core [None req-868279ab-6714-444e-bdaa-bc7fef8d3c33 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 16:31:56.946036 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 08 16:31:56.946036 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [parameters: {'id': '2e7ccd8cdefa4083bd17f978dc1d931a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 08 16:31:56.946036 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81999) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 16:31:56.966279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-868279ab-6714-444e-bdaa-bc7fef8d3c33 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 08 16:31:56.966279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:56.966279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 16:31:56.966279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 16:31:56.966279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 16:31:56.966279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 16:31:56.966279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 16:31:56.966279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 16:31:56.966279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 16:31:56.966279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application conn.query(q) Aug 08 16:31:56.966279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 16:31:56.966279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 16:31:56.966279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 16:31:56.966279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application result.read() Aug 08 16:31:56.966279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 16:31:56.967397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 16:31:56.967397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 16:31:56.967397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 16:31:56.967397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 16:31:56.967397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 16:31:56.967397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 16:31:56.967397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 16:31:56.967397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 16:31:56.967397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:56.967397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 16:31:56.967397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:56.967397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:56.967397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 16:31:56.967397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 16:31:56.967397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 08 16:31:56.967397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return ref.to_dict() Aug 08 16:31:56.967397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 16:31:56.967397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application next(self.gen) Aug 08 16:31:56.968817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 16:31:56.968817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application yield resource Aug 08 16:31:56.968817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 16:31:56.968817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application next(self.gen) Aug 08 16:31:56.968817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 16:31:56.968817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 16:31:56.968817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 16:31:56.968817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 16:31:56.968817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 16:31:56.968817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise self.value Aug 08 16:31:56.968817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 16:31:56.968817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 16:31:56.968817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 16:31:56.968817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application session.commit() Aug 08 16:31:56.968817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 16:31:56.968817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 16:31:56.968817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 16:31:56.969954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 16:31:56.969954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 16:31:56.969954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self.session.flush() Aug 08 16:31:56.969954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 16:31:56.969954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 16:31:56.969954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 16:31:56.969954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 16:31:56.969954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 16:31:56.969954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application compat.raise_( Aug 08 16:31:56.969954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 16:31:56.969954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception Aug 08 16:31:56.969954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 16:31:56.969954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 16:31:56.969954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 16:31:56.969954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 16:31:56.969954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 16:31:56.969954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 16:31:56.971305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 16:31:56.971305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 16:31:56.971305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 08 16:31:56.971305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 08 16:31:56.971305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 16:31:56.971305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 16:31:56.971305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 16:31:56.971305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 16:31:56.971305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 16:31:56.971305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 16:31:56.971305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 16:31:56.971305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 16:31:56.971305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 16:31:56.971305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 16:31:56.971305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 16:31:56.971305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception Aug 08 16:31:56.972268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 16:31:56.972268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 16:31:56.972268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 16:31:56.972268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 16:31:56.972268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 16:31:56.972268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 16:31:56.972268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 16:31:56.972268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application conn.query(q) Aug 08 16:31:56.972268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 16:31:56.972268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 16:31:56.972268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 16:31:56.972268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application result.read() Aug 08 16:31:56.972268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 16:31:56.972268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 16:31:56.972268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 16:31:56.972268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 16:31:56.973070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 16:31:56.973070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 16:31:56.973070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 16:31:56.973070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 16:31:56.973070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 16:31:56.973070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 08 16:31:56.973070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [parameters: {'id': '2e7ccd8cdefa4083bd17f978dc1d931a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 08 16:31:56.973070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 16:31:56.973070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:56.973070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:31:56.973070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:56.973070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:56.973070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:56.973070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:56.973070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:56.973070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:56.974117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:56.974117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:56.974117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:56.974117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:56.974117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:56.974117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:56.974117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 08 16:31:56.974117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 08 16:31:56.974117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:56.974117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:56.974117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 08 16:31:56.974117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 08 16:31:56.974117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 16:31:56.974117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 16:31:56.974117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 08 16:31:56.974117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:31:56.977114 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 245/478] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Aug 8 16:31:56 2022] POST /identity/v3/roles => generated 144 bytes in 237 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:56.983498 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad4ae6-0625-4f18-b1a2-43da50b920b3 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:57.047365 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:57.105472 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad4ae6-0625-4f18-b1a2-43da50b920b3 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:57.144281 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ed36f-4828-4756-8ca0-b3a7f7fb8a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:57.144921 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ed36f-4828-4756-8ca0-b3a7f7fb8a24 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:57.145199 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ed36f-4828-4756-8ca0-b3a7f7fb8a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:57.145472 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ed36f-4828-4756-8ca0-b3a7f7fb8a24 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:57.157006 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ed36f-4828-4756-8ca0-b3a7f7fb8a24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:57.163348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-a23ed36f-4828-4756-8ca0-b3a7f7fb8a24 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:57.163348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:57.163348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:57.163348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:57.163348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:57.163348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:57.163348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:57.163348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:57.163348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:57.163348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:57.163348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:57.163348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:57.163348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:57.163348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:57.163348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:57.163348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:57.165477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:57.165477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:57.165477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:57.165477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:57.165477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:57.165477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:57.165477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:57.165477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:57.165477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:57.165477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:57.165477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:57.165477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:57.165477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:57.165477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:57.165477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:57.165477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:57.167746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:57.167746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:57.167746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:57.167746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:57.167746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:57.167746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:57.167746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:57.167746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:57.167746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:57.167746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:57.167746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:31:57.167746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:57.167746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 234/479] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Aug 8 16:31:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:57.174596 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868279ab-6714-444e-bdaa-bc7fef8d3c33 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:57.305944 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868279ab-6714-444e-bdaa-bc7fef8d3c33 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:57.351467 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e82d81-8cb9-4039-add3-f3352125acb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:57.352397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e82d81-8cb9-4039-add3-f3352125acb4 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:57.352821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e82d81-8cb9-4039-add3-f3352125acb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:57.353269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e82d81-8cb9-4039-add3-f3352125acb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:57.354343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e82d81-8cb9-4039-add3-f3352125acb4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:57.354820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e82d81-8cb9-4039-add3-f3352125acb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:57.356433 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e82d81-8cb9-4039-add3-f3352125acb4 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:57.370962 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 246/480] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:31:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:58.577777 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f8ac6-1174-4dd3-8ef3-5e2929620247 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:58.578898 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f8ac6-1174-4dd3-8ef3-5e2929620247 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:58.579380 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f8ac6-1174-4dd3-8ef3-5e2929620247 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:58.581207 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 235/481] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:58.586087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14b3fe-8e2e-4fd9-b53c-1161a102bc5e None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:58.586975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14b3fe-8e2e-4fd9-b53c-1161a102bc5e None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:58.587484 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14b3fe-8e2e-4fd9-b53c-1161a102bc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:58.614156 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-4c14b3fe-8e2e-4fd9-b53c-1161a102bc5e None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:58.615747 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-4c14b3fe-8e2e-4fd9-b53c-1161a102bc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:58.618054 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:58.705595 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 247/482] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:58.736513 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9f8ac6-1174-4dd3-8ef3-5e2929620247 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:58.874408 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9f8ac6-1174-4dd3-8ef3-5e2929620247 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:58.890264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:58.930101 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951446bd-47e7-468f-866a-90df378d1266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:58.932436 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951446bd-47e7-468f-866a-90df378d1266 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:58.932943 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951446bd-47e7-468f-866a-90df378d1266 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:58.933428 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951446bd-47e7-468f-866a-90df378d1266 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:58.949465 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951446bd-47e7-468f-866a-90df378d1266 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:58.954829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-951446bd-47e7-468f-866a-90df378d1266 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:31:58.954829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:58.954829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:58.954829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:58.954829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:58.954829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:58.954829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:58.954829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:58.954829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:58.954829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:58.954829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:58.954829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:58.954829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:58.954829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:58.954829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:58.954829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:58.955733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:58.955733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:58.955733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:58.955733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:58.955733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:58.955733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:58.955733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:58.955733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:58.955733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:58.955733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:58.955733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:58.955733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:58.955733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:58.955733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:58.955733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:58.955733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:58.956541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:58.956541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:58.956541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:58.956541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:58.956541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:58.956541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:58.956541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:58.956541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:58.956541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:58.956541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:58.956541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:31:58.956541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:58.958865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 236/483] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:31:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:58.965676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14b3fe-8e2e-4fd9-b53c-1161a102bc5e None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:59.097944 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14b3fe-8e2e-4fd9-b53c-1161a102bc5e None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:59.142238 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb731d8-17dd-4757-967e-cc8f0ce5d37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:59.143219 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb731d8-17dd-4757-967e-cc8f0ce5d37c None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:59.143722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb731d8-17dd-4757-967e-cc8f0ce5d37c None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:59.144213 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb731d8-17dd-4757-967e-cc8f0ce5d37c None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:59.145087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb731d8-17dd-4757-967e-cc8f0ce5d37c None admin] RBAC: Adding query filter params (name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:59.145587 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb731d8-17dd-4757-967e-cc8f0ce5d37c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:59.147188 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb731d8-17dd-4757-967e-cc8f0ce5d37c None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:59.157658 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 248/484] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Aug 8 16:31:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:59.161675 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:59.162938 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951446bd-47e7-468f-866a-90df378d1266 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:59.238722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feaa118-147e-4840-83e7-18243b7bc5f4 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:59.239359 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feaa118-147e-4840-83e7-18243b7bc5f4 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:59.239894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feaa118-147e-4840-83e7-18243b7bc5f4 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:59.241799 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 249/485] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:31:59.249141 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326c6fb-378c-4106-ab90-639516ae82ce None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:59.249462 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326c6fb-378c-4106-ab90-639516ae82ce None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:59.250238 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326c6fb-378c-4106-ab90-639516ae82ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:59.274053 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-9326c6fb-378c-4106-ab90-639516ae82ce None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:59.276407 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-9326c6fb-378c-4106-ab90-639516ae82ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:59.313228 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951446bd-47e7-468f-866a-90df378d1266 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:59.355140 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 250/486] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:59.359773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122b3bf-b1a9-4097-b0fe-578e376cab7c None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:59.360054 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122b3bf-b1a9-4097-b0fe-578e376cab7c None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:59.360397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122b3bf-b1a9-4097-b0fe-578e376cab7c None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:59.361850 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 251/487] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:59.367838 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb94e3b-a05e-4734-a180-5df1f8f1b911 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:59.368112 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb94e3b-a05e-4734-a180-5df1f8f1b911 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:59.368378 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb94e3b-a05e-4734-a180-5df1f8f1b911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:59.373311 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f9462-aa4d-4e12-b5b9-2cc2137f5326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:59.373946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f9462-aa4d-4e12-b5b9-2cc2137f5326 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:59.374219 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f9462-aa4d-4e12-b5b9-2cc2137f5326 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:59.374483 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f9462-aa4d-4e12-b5b9-2cc2137f5326 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:59.388130 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f9462-aa4d-4e12-b5b9-2cc2137f5326 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:59.389868 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f9462-aa4d-4e12-b5b9-2cc2137f5326 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:59.406021 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-ccb94e3b-a05e-4734-a180-5df1f8f1b911 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:59.407197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-033f9462-aa4d-4e12-b5b9-2cc2137f5326 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:31:59.407197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:59.407197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:59.407197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:59.407197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:59.407197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:59.407197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:59.407197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:59.407197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:59.407197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:59.407197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:59.407197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:59.407197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:31:59.407197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:31:59.407197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:31:59.407197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:31:59.408183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:59.408183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:59.408183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:59.408183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:59.408183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:59.408183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:59.408183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:59.408183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:59.408183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:59.408183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:59.408183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:59.408183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:59.408183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:59.408183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:59.408183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:59.408183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:59.408183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:31:59.409142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:31:59.409142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:31:59.409142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:59.409324 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ccb94e3b-a05e-4734-a180-5df1f8f1b911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:59.409404 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 237/488] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:31:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:59.409987 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f9462-aa4d-4e12-b5b9-2cc2137f5326 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:59.439139 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:59.498757 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 252/489] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:59.499946 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb94e3b-a05e-4734-a180-5df1f8f1b911 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:59.527086 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f9462-aa4d-4e12-b5b9-2cc2137f5326 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:59.562607 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:59.575543 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a930ed-5f5b-406b-875a-6a07a71e2fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:59.576185 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a930ed-5f5b-406b-875a-6a07a71e2fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:59.576460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a930ed-5f5b-406b-875a-6a07a71e2fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:59.576729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a930ed-5f5b-406b-875a-6a07a71e2fe8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:59.587501 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a930ed-5f5b-406b-875a-6a07a71e2fe8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:59.592229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-00a930ed-5f5b-406b-875a-6a07a71e2fe8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:31:59.592229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:59.592229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:59.592229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:59.592229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:59.592229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:59.592229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:59.592229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:59.592229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:59.592229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:59.592229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:59.592229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:59.592229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:59.592229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:59.592229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:59.592229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:59.593413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:59.593413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:59.593413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:59.593413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:59.593413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:59.593413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:59.593413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:59.593413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:59.593413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:59.593413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:59.593413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:59.593413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:59.593413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:59.593413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:59.593413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:59.593413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:59.594248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:59.594248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:59.594248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:59.594248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:59.594248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:59.594248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:59.594248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:59.594248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:59.594248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:59.594248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:59.594248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:31:59.594248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:59.594248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 238/490] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:31:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:59.594958 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a3ee2-7602-4a40-8431-8ba14b84980e None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:59.595350 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a3ee2-7602-4a40-8431-8ba14b84980e None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:59.595350 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a3ee2-7602-4a40-8431-8ba14b84980e None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:59.596719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 239/491] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:31:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:31:59.599417 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a3ee2-7602-4a40-8431-8ba14b84980e None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:59.615528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb94e3b-a05e-4734-a180-5df1f8f1b911 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:59.656855 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ad720-15c9-4ad7-b8b2-1b6471854bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:59.657389 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ad720-15c9-4ad7-b8b2-1b6471854bdd None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:59.657661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ad720-15c9-4ad7-b8b2-1b6471854bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:59.658273 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ad720-15c9-4ad7-b8b2-1b6471854bdd None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:59.658635 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ad720-15c9-4ad7-b8b2-1b6471854bdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:59.658997 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ad720-15c9-4ad7-b8b2-1b6471854bdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:59.660438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ad720-15c9-4ad7-b8b2-1b6471854bdd None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:59.674117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 253/492] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:59.674117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ad720-15c9-4ad7-b8b2-1b6471854bdd None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:59.696900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:59.718134 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a3ee2-7602-4a40-8431-8ba14b84980e None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:59.755536 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f237e66-40fe-47a3-977c-42634c0c234d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:59.756167 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f237e66-40fe-47a3-977c-42634c0c234d None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:59.756446 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f237e66-40fe-47a3-977c-42634c0c234d None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:59.756713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f237e66-40fe-47a3-977c-42634c0c234d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:59.767548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f237e66-40fe-47a3-977c-42634c0c234d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:31:59.772158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-4f237e66-40fe-47a3-977c-42634c0c234d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:31:59.772158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:31:59.772158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:31:59.772158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:31:59.772158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:31:59.772158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:31:59.772158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:31:59.772158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:31:59.772158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:31:59.772158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:31:59.772158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:31:59.772158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:31:59.772158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:31:59.772158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:31:59.772158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:31:59.772158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:31:59.785243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:31:59.785243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:31:59.785243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:31:59.785243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:31:59.785243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:31:59.785243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:31:59.785243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:31:59.785243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:31:59.785243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:31:59.785243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:31:59.785243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:31:59.785243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:31:59.785243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:31:59.785243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:31:59.785243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:31:59.785243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:31:59.786080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:31:59.786080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:31:59.786080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:31:59.786080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:31:59.786080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:31:59.786080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:31:59.786080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:31:59.786080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:31:59.786080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:31:59.786080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:31:59.786080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:31:59.786080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:31:59.786080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 240/493] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:31:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:31:59.786080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33e3f9-0968-46bb-8ce9-fe8b4c315996 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:59.786866 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33e3f9-0968-46bb-8ce9-fe8b4c315996 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:59.786866 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33e3f9-0968-46bb-8ce9-fe8b4c315996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:59.792990 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-7b33e3f9-0968-46bb-8ce9-fe8b4c315996 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:31:59.794344 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-7b33e3f9-0968-46bb-8ce9-fe8b4c315996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:31:59.797842 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ad720-15c9-4ad7-b8b2-1b6471854bdd None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:59.819094 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:31:59.833386 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd803fc6-7b25-4373-85a9-1ac651329401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:31:59.833998 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd803fc6-7b25-4373-85a9-1ac651329401 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:31:59.834277 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd803fc6-7b25-4373-85a9-1ac651329401 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:31:59.834544 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd803fc6-7b25-4373-85a9-1ac651329401 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:31:59.835228 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd803fc6-7b25-4373-85a9-1ac651329401 None admin] RBAC: Adding query filter params (name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:31:59.835524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd803fc6-7b25-4373-85a9-1ac651329401 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:31:59.836952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd803fc6-7b25-4373-85a9-1ac651329401 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:31:59.845760 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 254/494] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Aug 8 16:31:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:31:59.846486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd803fc6-7b25-4373-85a9-1ac651329401 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:59.886704 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 241/495] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:31:59.887811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b33e3f9-0968-46bb-8ce9-fe8b4c315996 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:31:59.977620 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd803fc6-7b25-4373-85a9-1ac651329401 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:31:59.977866 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:00.011597 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b33e3f9-0968-46bb-8ce9-fe8b4c315996 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:00.018372 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9c934-b3ba-49a6-b066-eecd0630b297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:00.019050 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9c934-b3ba-49a6-b066-eecd0630b297 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:00.019332 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9c934-b3ba-49a6-b066-eecd0630b297 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:00.019604 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9c934-b3ba-49a6-b066-eecd0630b297 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:00.028014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b9c934-b3ba-49a6-b066-eecd0630b297 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:00.029489 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b9c934-b3ba-49a6-b066-eecd0630b297 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:00.042984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-63b9c934-b3ba-49a6-b066-eecd0630b297 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:32:00.042984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:00.042984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:00.042984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:00.042984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:00.042984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:00.042984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:00.042984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:00.042984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:00.042984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:00.042984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:00.042984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:00.042984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:00.042984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:00.042984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:00.042984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:00.043979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:00.043979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:00.043979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:00.043979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:00.043979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:00.043979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:00.043979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:00.043979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:00.043979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:00.043979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:00.043979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:00.043979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:00.043979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:00.043979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:00.043979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:00.043979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:00.043979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:00.043979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:00.044896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:00.044896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:00.044896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:00.044896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:00.044896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:00.044896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:00.044896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:00.044896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:00.044896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:32:00.044896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:00.044896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 255/496] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Aug 8 16:31:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:00.045503 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9c934-b3ba-49a6-b066-eecd0630b297 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:00.058990 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333224aa-07ba-4262-b6f8-0499d0658bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:00.059614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333224aa-07ba-4262-b6f8-0499d0658bea None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:00.059889 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333224aa-07ba-4262-b6f8-0499d0658bea None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:00.060153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333224aa-07ba-4262-b6f8-0499d0658bea None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:00.060861 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333224aa-07ba-4262-b6f8-0499d0658bea None admin] RBAC: Adding query filter params (name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:00.061163 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333224aa-07ba-4262-b6f8-0499d0658bea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:00.062603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333224aa-07ba-4262-b6f8-0499d0658bea None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:00.077595 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 242/497] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Aug 8 16:31:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:00.078393 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333224aa-07ba-4262-b6f8-0499d0658bea None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:00.098107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:00.166498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9c934-b3ba-49a6-b066-eecd0630b297 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:00.201818 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333224aa-07ba-4262-b6f8-0499d0658bea None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:00.207229 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a2b1c-e8b1-4c24-a56a-6fbd4cbab0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:00.207873 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a2b1c-e8b1-4c24-a56a-6fbd4cbab0fe None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:00.208146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a2b1c-e8b1-4c24-a56a-6fbd4cbab0fe None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:00.208412 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a2b1c-e8b1-4c24-a56a-6fbd4cbab0fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:00.220482 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a2b1c-e8b1-4c24-a56a-6fbd4cbab0fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:00.221957 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a2b1c-e8b1-4c24-a56a-6fbd4cbab0fe None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:00.237305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-dd2a2b1c-e8b1-4c24-a56a-6fbd4cbab0fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:00.237305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:00.237305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:00.237305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:00.237305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:00.237305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:00.237305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:00.237305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:00.237305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:00.237305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:00.237305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:00.237305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:00.237305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:00.237305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:00.237305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:00.237305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:00.238478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:00.238478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:00.238478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:00.238478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:00.238478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:00.238478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:00.238478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:00.238478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:00.238478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:00.238478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:00.238478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:00.238478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:00.238478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:00.238478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:00.238478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:00.238478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:00.238478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:00.239560 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:00.239560 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:00.239560 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:00.239560 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 256/498] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:00.239560 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a2b1c-e8b1-4c24-a56a-6fbd4cbab0fe None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:00.242290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c6999-f939-43a7-9c83-a472ac388f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:00.242290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c6999-f939-43a7-9c83-a472ac388f12 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:00.242290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c6999-f939-43a7-9c83-a472ac388f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:00.242290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c6999-f939-43a7-9c83-a472ac388f12 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:00.245549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:00.255576 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18c6999-f939-43a7-9c83-a472ac388f12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:32:00.261259 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-c18c6999-f939-43a7-9c83-a472ac388f12 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:00.261259 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:00.261259 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:00.261259 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:00.261259 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:00.261259 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:00.261259 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:00.261259 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:00.261259 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:00.261259 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:00.261259 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:00.261259 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:00.261259 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:32:00.261259 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:32:00.261259 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:32:00.261259 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:32:00.267142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:32:00.267142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:32:00.267142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:32:00.267142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:32:00.267142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:00.267142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:00.267142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:00.267142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:00.267142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:00.267142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:00.267142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:00.267142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:00.267142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:00.267142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:00.267142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:00.267142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:00.268291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:00.268291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:00.268291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:00.268291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:00.268291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:32:00.268291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:32:00.268291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:32:00.268291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:32:00.268291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:32:00.268291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:32:00.268291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:00.268291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:00.268291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 243/499] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:32:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:00.268291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c6999-f939-43a7-9c83-a472ac388f12 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:00.344909 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:00.351959 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a2b1c-e8b1-4c24-a56a-6fbd4cbab0fe None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:00.375348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c6999-f939-43a7-9c83-a472ac388f12 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:00.394413 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711d302-f4be-4034-873f-11b6cec5f757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:00.395117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711d302-f4be-4034-873f-11b6cec5f757 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:00.395398 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711d302-f4be-4034-873f-11b6cec5f757 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:00.395671 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711d302-f4be-4034-873f-11b6cec5f757 None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:00.396392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711d302-f4be-4034-873f-11b6cec5f757 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:00.396694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711d302-f4be-4034-873f-11b6cec5f757 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:00.398146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711d302-f4be-4034-873f-11b6cec5f757 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:00.411967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91336eb2-2b9e-4d45-9671-d8f615ae6eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:00.412589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91336eb2-2b9e-4d45-9671-d8f615ae6eac None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:00.412868 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91336eb2-2b9e-4d45-9671-d8f615ae6eac None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:00.413131 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91336eb2-2b9e-4d45-9671-d8f615ae6eac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:00.423797 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91336eb2-2b9e-4d45-9671-d8f615ae6eac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:00.425272 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91336eb2-2b9e-4d45-9671-d8f615ae6eac None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:00.437550 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 257/500] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Aug 8 16:32:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:00.438843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed272a7-94ab-4eae-a29c-679df9a2ebd9 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:00.439130 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed272a7-94ab-4eae-a29c-679df9a2ebd9 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:00.439403 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed272a7-94ab-4eae-a29c-679df9a2ebd9 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:00.440140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-91336eb2-2b9e-4d45-9671-d8f615ae6eac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:00.440140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:00.440140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:00.440140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:00.440140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:00.440140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:00.440140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:00.440140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:00.440140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:00.440140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:00.440140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:00.440140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:00.440140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:00.440140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:00.440140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:00.440140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:00.441011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:00.441011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:00.441011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:00.441011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:00.441011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:00.441011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:00.441011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:00.441011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:00.441011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:00.441011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:00.441011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:00.441011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:00.441011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:00.441011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:00.441011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:00.441011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:00.441011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:00.441871 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:00.441871 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:00.441871 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:00.442693 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 258/501] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:00.443449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed272a7-94ab-4eae-a29c-679df9a2ebd9 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:00.445667 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 244/502] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:00.446425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91336eb2-2b9e-4d45-9671-d8f615ae6eac None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:00.494751 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:00.546091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed272a7-94ab-4eae-a29c-679df9a2ebd9 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:00.549167 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91336eb2-2b9e-4d45-9671-d8f615ae6eac None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:00.580770 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1327a6e-0b38-408a-9787-5586c5de6d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:00.581375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1327a6e-0b38-408a-9787-5586c5de6d5c None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:00.581653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1327a6e-0b38-408a-9787-5586c5de6d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:00.581918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1327a6e-0b38-408a-9787-5586c5de6d5c None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:00.582556 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1327a6e-0b38-408a-9787-5586c5de6d5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:00.582874 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1327a6e-0b38-408a-9787-5586c5de6d5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:00.583592 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201e739-3c53-4b99-b3aa-4374281a1baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:00.584188 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201e739-3c53-4b99-b3aa-4374281a1baa None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:00.584275 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1327a6e-0b38-408a-9787-5586c5de6d5c None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:00.584463 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201e739-3c53-4b99-b3aa-4374281a1baa None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:00.584729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201e739-3c53-4b99-b3aa-4374281a1baa None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:00.585387 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201e739-3c53-4b99-b3aa-4374281a1baa None admin] RBAC: Adding query filter params (name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:00.585685 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201e739-3c53-4b99-b3aa-4374281a1baa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:00.587091 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201e739-3c53-4b99-b3aa-4374281a1baa None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:00.595349 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 245/503] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Aug 8 16:32:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:00.596171 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201e739-3c53-4b99-b3aa-4374281a1baa None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:00.598601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 259/504] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:00.598601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236e9ea-3437-43b0-b880-a4ad8ad53bf2 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:00.598601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236e9ea-3437-43b0-b880-a4ad8ad53bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:00.598601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236e9ea-3437-43b0-b880-a4ad8ad53bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:00.617837 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-1236e9ea-3437-43b0-b880-a4ad8ad53bf2 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:00.619588 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-1236e9ea-3437-43b0-b880-a4ad8ad53bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:00.627327 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:00.689288 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 260/505] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:32:00.690151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236e9ea-3437-43b0-b880-a4ad8ad53bf2 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:00.703516 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201e739-3c53-4b99-b3aa-4374281a1baa None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:00.712592 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:00.737969 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e60be4-59d6-44c2-b14e-39bbad3ed9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:00.738601 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e60be4-59d6-44c2-b14e-39bbad3ed9d2 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:00.738861 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e60be4-59d6-44c2-b14e-39bbad3ed9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:00.739122 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e60be4-59d6-44c2-b14e-39bbad3ed9d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:00.807452 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e60be4-59d6-44c2-b14e-39bbad3ed9d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:00.808389 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236e9ea-3437-43b0-b880-a4ad8ad53bf2 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:00.808845 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e60be4-59d6-44c2-b14e-39bbad3ed9d2 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:00.841806 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48413f48-0335-405a-833d-26727dba878b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:00.842406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48413f48-0335-405a-833d-26727dba878b None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:00.843005 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48413f48-0335-405a-833d-26727dba878b None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:00.843254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48413f48-0335-405a-833d-26727dba878b None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:00.843894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48413f48-0335-405a-833d-26727dba878b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:00.844187 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48413f48-0335-405a-833d-26727dba878b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:00.845546 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48413f48-0335-405a-833d-26727dba878b None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:00.857323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 261/506] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:00.858144 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48413f48-0335-405a-833d-26727dba878b None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:00.863361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:00.907322 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 246/507] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:32:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:32:00.908296 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e60be4-59d6-44c2-b14e-39bbad3ed9d2 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:00.919431 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:00.964484 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48413f48-0335-405a-833d-26727dba878b None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:00.999573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a22be-e124-4fcb-8410-cc76d6987a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:01.000208 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a22be-e124-4fcb-8410-cc76d6987a69 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:01.000490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a22be-e124-4fcb-8410-cc76d6987a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:01.000761 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a22be-e124-4fcb-8410-cc76d6987a69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:01.010937 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e60be4-59d6-44c2-b14e-39bbad3ed9d2 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:01.011609 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a22be-e124-4fcb-8410-cc76d6987a69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:01.013081 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a22be-e124-4fcb-8410-cc76d6987a69 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:01.027214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-674a22be-e124-4fcb-8410-cc76d6987a69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:01.027214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:01.027214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:01.027214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:01.027214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:01.027214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:01.027214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:01.027214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:01.027214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:01.027214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:01.027214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:01.027214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:01.027214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:01.027214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:01.027214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:01.027214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:01.028305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:01.028305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:01.028305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:01.028305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:01.028305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:01.028305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:01.028305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:01.028305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:01.028305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:01.028305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:01.028305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:01.028305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:01.028305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:01.028305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:01.028305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:01.028305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:01.028305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:01.029175 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:01.029175 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:01.029175 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:01.029175 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 262/508] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:01.029175 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a22be-e124-4fcb-8410-cc76d6987a69 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:01.045400 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ad90f-3a4d-496a-a83b-ddc51bb6b6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:01.046023 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ad90f-3a4d-496a-a83b-ddc51bb6b6ed None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:01.046293 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ad90f-3a4d-496a-a83b-ddc51bb6b6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:01.046553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ad90f-3a4d-496a-a83b-ddc51bb6b6ed None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:01.053450 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ad90f-3a4d-496a-a83b-ddc51bb6b6ed None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:01.054895 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ad90f-3a4d-496a-a83b-ddc51bb6b6ed None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:01.068340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-681ad90f-3a4d-496a-a83b-ddc51bb6b6ed None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 16:32:01.068340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:01.068340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:01.068340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:01.068340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:01.068340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:01.068340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:01.068340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:01.068340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:01.068340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:01.068340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:01.068340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:01.068340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:01.068340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:01.068340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:01.068340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:01.069479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:01.069479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:01.069479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:01.069479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:01.069479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:01.069479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:01.069479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:01.069479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:01.069479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:01.069479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:01.069479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:01.069479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:01.069479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:01.069479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:01.069479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:01.069479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:01.069479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:01.069479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:01.070408 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:01.070408 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:01.070408 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:01.070408 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:01.070408 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:01.070408 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:01.070408 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:01.070408 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:01.070408 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 16:32:01.070408 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:01.070408 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 247/509] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:32:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:01.071166 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ad90f-3a4d-496a-a83b-ddc51bb6b6ed None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:01.088914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:01.134395 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a22be-e124-4fcb-8410-cc76d6987a69 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:01.135984 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:01.167935 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4c85f-3146-44b1-a9f7-54246159b060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:01.168562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4c85f-3146-44b1-a9f7-54246159b060 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:01.168839 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4c85f-3146-44b1-a9f7-54246159b060 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:01.169111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4c85f-3146-44b1-a9f7-54246159b060 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:01.171848 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ad90f-3a4d-496a-a83b-ddc51bb6b6ed None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:01.179899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4c85f-3146-44b1-a9f7-54246159b060 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:01.182087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4c85f-3146-44b1-a9f7-54246159b060 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:01.195999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-12c4c85f-3146-44b1-a9f7-54246159b060 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:01.195999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:01.195999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:01.195999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:01.195999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:01.195999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:01.195999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:01.195999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:01.195999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:01.195999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:01.195999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:01.195999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:01.195999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:01.195999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:01.195999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:01.195999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:01.197205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:01.197205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:01.197205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:01.197205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:01.197205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:01.197205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:01.197205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:01.197205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:01.197205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:01.197205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:01.197205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:01.197205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:01.197205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:01.197205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:01.197205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:01.197205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:01.197205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:01.198082 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:01.198082 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:01.198082 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:01.198082 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 263/510] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:01.198082 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4c85f-3146-44b1-a9f7-54246159b060 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:01.215144 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff1d64-29a7-45ae-bb42-dd56445e5adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:01.215836 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff1d64-29a7-45ae-bb42-dd56445e5adb None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:01.216130 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff1d64-29a7-45ae-bb42-dd56445e5adb None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:01.216398 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff1d64-29a7-45ae-bb42-dd56445e5adb None admin] PATH_INFO: `/v3/users/glance` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:01.224360 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff1d64-29a7-45ae-bb42-dd56445e5adb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:01.225815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff1d64-29a7-45ae-bb42-dd56445e5adb None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:01.238076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-3aff1d64-29a7-45ae-bb42-dd56445e5adb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 16:32:01.238076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:01.238076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:01.238076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:01.238076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:01.238076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:01.238076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:01.238076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:01.238076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:01.238076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:01.238076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:01.238076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:01.238076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:01.238076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:01.238076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:01.238076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:01.239068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:01.239068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:01.239068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:01.239068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:01.239068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:01.239068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:01.239068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:01.239068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:01.239068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:01.239068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:01.239068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:01.239068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:01.239068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:01.239068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:01.239068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:01.239068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:01.239068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:01.239068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:01.239985 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:01.239985 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:01.239985 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:01.239985 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:01.239985 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:01.239985 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:01.239985 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:01.239985 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:01.239985 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 16:32:01.239985 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:01.239985 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 248/511] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:32:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:01.239985 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff1d64-29a7-45ae-bb42-dd56445e5adb None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:01.306457 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4c85f-3146-44b1-a9f7-54246159b060 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:01.313898 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:01.340251 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff1d64-29a7-45ae-bb42-dd56445e5adb None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:01.345562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1818d2d-3581-48ac-bc2e-6d91a0e7ca9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:01.346204 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1818d2d-3581-48ac-bc2e-6d91a0e7ca9a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:01.346483 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1818d2d-3581-48ac-bc2e-6d91a0e7ca9a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:01.346792 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1818d2d-3581-48ac-bc2e-6d91a0e7ca9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:01.358859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1818d2d-3581-48ac-bc2e-6d91a0e7ca9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:01.360363 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1818d2d-3581-48ac-bc2e-6d91a0e7ca9a None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:01.368933 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:01.375109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-b1818d2d-3581-48ac-bc2e-6d91a0e7ca9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:01.375109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:01.375109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:01.375109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:01.375109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:01.375109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:01.375109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:01.375109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:01.375109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:01.375109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:01.375109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:01.375109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:01.375109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:01.375109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:01.375109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:01.375109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:01.378360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:01.378360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:01.378360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:01.378360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:01.378360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:01.378360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:01.378360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:01.378360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:01.378360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:01.378360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:01.378360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:01.378360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:01.378360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:01.378360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:01.378360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:01.378360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:01.378360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:01.379473 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:01.379473 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:01.379473 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:01.379473 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 264/512] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:01.379473 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1818d2d-3581-48ac-bc2e-6d91a0e7ca9a None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:01.379761 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fef50b-d6f2-40c9-86de-e4af95ceb36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:01.379761 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fef50b-d6f2-40c9-86de-e4af95ceb36a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:01.379761 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fef50b-d6f2-40c9-86de-e4af95ceb36a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:01.379761 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fef50b-d6f2-40c9-86de-e4af95ceb36a None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:01.380014 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fef50b-d6f2-40c9-86de-e4af95ceb36a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:01.380014 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fef50b-d6f2-40c9-86de-e4af95ceb36a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:01.380841 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fef50b-d6f2-40c9-86de-e4af95ceb36a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:01.394197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 249/513] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:01.395295 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fef50b-d6f2-40c9-86de-e4af95ceb36a None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:01.479622 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1818d2d-3581-48ac-bc2e-6d91a0e7ca9a None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:01.497399 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fef50b-d6f2-40c9-86de-e4af95ceb36a None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:01.515634 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b12f7-2ca0-48ce-ac9f-d85af74f3044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:01.516247 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b12f7-2ca0-48ce-ac9f-d85af74f3044 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:01.516522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b12f7-2ca0-48ce-ac9f-d85af74f3044 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:01.516791 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b12f7-2ca0-48ce-ac9f-d85af74f3044 None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:01.517517 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b12f7-2ca0-48ce-ac9f-d85af74f3044 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:01.517818 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b12f7-2ca0-48ce-ac9f-d85af74f3044 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:01.519275 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b12f7-2ca0-48ce-ac9f-d85af74f3044 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:01.531081 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d79d37-ec01-4ca7-9aba-07b9a7cb2ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:01.531678 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d79d37-ec01-4ca7-9aba-07b9a7cb2ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:01.531951 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d79d37-ec01-4ca7-9aba-07b9a7cb2ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:01.532214 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d79d37-ec01-4ca7-9aba-07b9a7cb2ab4 None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:01.532839 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d79d37-ec01-4ca7-9aba-07b9a7cb2ab4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:01.533138 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d79d37-ec01-4ca7-9aba-07b9a7cb2ab4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:01.534493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d79d37-ec01-4ca7-9aba-07b9a7cb2ab4 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:01.536547 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 265/514] 10.222.0.13 () {56 vars in 1139 bytes} [Mon Aug 8 16:32:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:01.537367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b12f7-2ca0-48ce-ac9f-d85af74f3044 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:01.550747 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 250/515] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:01.551651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d79d37-ec01-4ca7-9aba-07b9a7cb2ab4 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:01.561393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:01.601084 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:01.643182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b12f7-2ca0-48ce-ac9f-d85af74f3044 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:01.653259 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d79d37-ec01-4ca7-9aba-07b9a7cb2ab4 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:01.679064 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08750cf-d740-476a-bba2-cbea39b43d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:01.679853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08750cf-d740-476a-bba2-cbea39b43d19 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:01.680236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08750cf-d740-476a-bba2-cbea39b43d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:01.680654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08750cf-d740-476a-bba2-cbea39b43d19 None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:01.682393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08750cf-d740-476a-bba2-cbea39b43d19 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:01.682812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08750cf-d740-476a-bba2-cbea39b43d19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:01.684641 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08750cf-d740-476a-bba2-cbea39b43d19 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:01.688790 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a098012-a33b-495e-85d2-0ed7e83bc647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:01.689387 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a098012-a33b-495e-85d2-0ed7e83bc647 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:01.689661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a098012-a33b-495e-85d2-0ed7e83bc647 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:01.689925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a098012-a33b-495e-85d2-0ed7e83bc647 None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:01.690555 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a098012-a33b-495e-85d2-0ed7e83bc647 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:01.690879 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a098012-a33b-495e-85d2-0ed7e83bc647 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:01.692229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a098012-a33b-495e-85d2-0ed7e83bc647 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:01.703496 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 266/516] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Aug 8 16:32:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:01.708929 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 251/517] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:01.709074 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08750cf-d740-476a-bba2-cbea39b43d19 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:01.709162 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a098012-a33b-495e-85d2-0ed7e83bc647 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:01.780871 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:01.806823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08750cf-d740-476a-bba2-cbea39b43d19 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:01.809281 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a098012-a33b-495e-85d2-0ed7e83bc647 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:01.818863 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:01.851547 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da108b-a2e2-4f6a-ab91-c6cdcc8461cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:01.852457 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da108b-a2e2-4f6a-ab91-c6cdcc8461cf None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:01.853440 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da108b-a2e2-4f6a-ab91-c6cdcc8461cf None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:01.854029 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da108b-a2e2-4f6a-ab91-c6cdcc8461cf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:01.856135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc3804-aba0-47b3-aa41-e0bb522e4db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:01.857076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc3804-aba0-47b3-aa41-e0bb522e4db9 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:01.857518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc3804-aba0-47b3-aa41-e0bb522e4db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:01.857949 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc3804-aba0-47b3-aa41-e0bb522e4db9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:01.865030 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23da108b-a2e2-4f6a-ab91-c6cdcc8461cf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:01.867532 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23da108b-a2e2-4f6a-ab91-c6cdcc8461cf None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:01.887151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-23da108b-a2e2-4f6a-ab91-c6cdcc8461cf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 16:32:01.887151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:01.887151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:01.887151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:01.887151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:01.887151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:01.887151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:01.887151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:01.887151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:01.887151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:01.887151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:01.887151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:01.887151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:01.887151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:01.887151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:01.887151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:01.888260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:01.888260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:01.888260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:01.888260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:01.888260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:01.888260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:01.888260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:01.888260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:01.888260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:01.888260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:01.888260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:01.888260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:01.888260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:01.888260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:01.888260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:01.888260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:01.888260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:01.888260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:01.889367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:01.889367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:01.889367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:01.889367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:01.889367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:01.889367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:01.889367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:01.889367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:01.889367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 16:32:01.889367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:01.891908 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 267/518] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:32:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:01.891908 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da108b-a2e2-4f6a-ab91-c6cdcc8461cf None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:01.943062 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc3804-aba0-47b3-aa41-e0bb522e4db9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:01.944543 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc3804-aba0-47b3-aa41-e0bb522e4db9 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:01.996076 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da108b-a2e2-4f6a-ab91-c6cdcc8461cf None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:02.030876 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd110d55-580f-4f0b-9545-d63629d0eb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:02.031481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd110d55-580f-4f0b-9545-d63629d0eb33 None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:02.031754 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd110d55-580f-4f0b-9545-d63629d0eb33 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:02.032020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd110d55-580f-4f0b-9545-d63629d0eb33 None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:02.032657 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd110d55-580f-4f0b-9545-d63629d0eb33 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:02.034096 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd110d55-580f-4f0b-9545-d63629d0eb33 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:02.042536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:02.045925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 252/519] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:32:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:32:02.046808 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc3804-aba0-47b3-aa41-e0bb522e4db9 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:02.058691 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:02.130723 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 268/520] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Aug 8 16:32:01 2022] POST /identity/v3/users => generated 247 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:32:02.131901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd110d55-580f-4f0b-9545-d63629d0eb33 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:02.170682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc3804-aba0-47b3-aa41-e0bb522e4db9 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:02.223081 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8bf34-e1a0-4c01-81db-f5aaf1ab8a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:02.224505 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8bf34-e1a0-4c01-81db-f5aaf1ab8a40 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:02.224759 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8bf34-e1a0-4c01-81db-f5aaf1ab8a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:02.225403 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8bf34-e1a0-4c01-81db-f5aaf1ab8a40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:02.275706 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd110d55-580f-4f0b-9545-d63629d0eb33 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:02.307873 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8bf34-e1a0-4c01-81db-f5aaf1ab8a40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:02.309352 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8bf34-e1a0-4c01-81db-f5aaf1ab8a40 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:02.322275 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:02.331946 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70097fc1-6b5c-43d2-858d-069e81b0ccc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:02.333081 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70097fc1-6b5c-43d2-858d-069e81b0ccc4 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:02.334349 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70097fc1-6b5c-43d2-858d-069e81b0ccc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:02.334896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70097fc1-6b5c-43d2-858d-069e81b0ccc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:02.347748 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70097fc1-6b5c-43d2-858d-069e81b0ccc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:02.349502 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70097fc1-6b5c-43d2-858d-069e81b0ccc4 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:02.365392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-70097fc1-6b5c-43d2-858d-069e81b0ccc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:02.365392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:02.365392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:02.365392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:02.365392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:02.365392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:02.365392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:02.365392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:02.365392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:02.365392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:02.365392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:02.365392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:02.365392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:02.365392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:02.365392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:02.366453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:02.366453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:02.366453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:02.366453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:02.366453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:02.366453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:02.366453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:02.366453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:02.366453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:02.366453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:02.366453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:02.366453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:02.366453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:02.366453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:02.366453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:02.366453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:02.366453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:02.367357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:02.367357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:02.367357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:02.367357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:02.367357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:02.367357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:02.367357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:02.367357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:02.367789 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 269/521] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:02.370661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70097fc1-6b5c-43d2-858d-069e81b0ccc4 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:02.443561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:02.451476 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 253/522] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:32:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:32:02.452284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8bf34-e1a0-4c01-81db-f5aaf1ab8a40 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:02.513033 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70097fc1-6b5c-43d2-858d-069e81b0ccc4 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:02.551199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71182604-513c-4c56-a55a-08119fe8a003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:02.552139 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71182604-513c-4c56-a55a-08119fe8a003 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:02.552704 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71182604-513c-4c56-a55a-08119fe8a003 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:02.552972 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71182604-513c-4c56-a55a-08119fe8a003 None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:02.553696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71182604-513c-4c56-a55a-08119fe8a003 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:02.553996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71182604-513c-4c56-a55a-08119fe8a003 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:02.555551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71182604-513c-4c56-a55a-08119fe8a003 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:02.565119 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8bf34-e1a0-4c01-81db-f5aaf1ab8a40 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:02.575596 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:02.582503 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 270/523] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Aug 8 16:32:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:02.582503 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71182604-513c-4c56-a55a-08119fe8a003 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:02.611032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399acfd-d4c6-4384-827c-8fb948ffbdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:02.611852 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399acfd-d4c6-4384-827c-8fb948ffbdae None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:02.612129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399acfd-d4c6-4384-827c-8fb948ffbdae None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:02.612395 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399acfd-d4c6-4384-827c-8fb948ffbdae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:02.624654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8399acfd-d4c6-4384-827c-8fb948ffbdae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:02.626124 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8399acfd-d4c6-4384-827c-8fb948ffbdae None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:02.644123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-8399acfd-d4c6-4384-827c-8fb948ffbdae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:02.644123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:02.644123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:02.644123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:02.644123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:02.644123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:02.644123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:02.644123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:02.644123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:02.644123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:02.644123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:02.644123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:02.644123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:02.644123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:02.644123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:02.644123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:02.645565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:02.645565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:02.645565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:02.645565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:02.645565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:02.645565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:02.645565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:02.645565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:02.645565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:02.645565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:02.645565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:02.645565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:02.645565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:02.645565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:02.645565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:02.645565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:02.645565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:02.646471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:02.646471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:02.646471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:02.646471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 254/524] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:02.646471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399acfd-d4c6-4384-827c-8fb948ffbdae None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:02.707118 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71182604-513c-4c56-a55a-08119fe8a003 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:02.719448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:02.749405 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a41f6b-d80a-48a8-aea9-e51ee5ffd579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:02.750376 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a41f6b-d80a-48a8-aea9-e51ee5ffd579 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:02.750888 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a41f6b-d80a-48a8-aea9-e51ee5ffd579 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:02.751359 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a41f6b-d80a-48a8-aea9-e51ee5ffd579 None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:02.752236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a41f6b-d80a-48a8-aea9-e51ee5ffd579 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:02.752666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a41f6b-d80a-48a8-aea9-e51ee5ffd579 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:02.754658 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a41f6b-d80a-48a8-aea9-e51ee5ffd579 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:02.761622 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399acfd-d4c6-4384-827c-8fb948ffbdae None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:02.780155 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 271/525] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:02.780155 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a41f6b-d80a-48a8-aea9-e51ee5ffd579 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:02.805662 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d305c-6f3e-4fc6-8c55-675ddded9f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:02.807556 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d305c-6f3e-4fc6-8c55-675ddded9f1a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:02.807832 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d305c-6f3e-4fc6-8c55-675ddded9f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:02.808097 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d305c-6f3e-4fc6-8c55-675ddded9f1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:02.820871 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d305c-6f3e-4fc6-8c55-675ddded9f1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:02.823213 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d305c-6f3e-4fc6-8c55-675ddded9f1a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:02.838379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-730d305c-6f3e-4fc6-8c55-675ddded9f1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:02.838379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:02.838379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:02.838379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:02.838379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:02.838379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:02.838379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:02.838379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:02.838379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:02.838379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:02.838379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:02.838379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:02.838379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:02.838379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:02.838379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:02.838379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:02.839634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:02.839634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:02.839634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:02.839634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:02.839634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:02.839634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:02.839634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:02.839634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:02.839634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:02.839634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:02.839634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:02.839634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:02.839634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:02.839634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:02.839634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:02.839634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:02.839634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:02.840544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:02.840544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:02.840544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:02.840544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 255/526] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:02.840544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d305c-6f3e-4fc6-8c55-675ddded9f1a None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:02.873645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:02.906443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a41f6b-d80a-48a8-aea9-e51ee5ffd579 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:02.947450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83407ba-c7be-43b2-a938-22c49e761a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:02.947903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83407ba-c7be-43b2-a938-22c49e761a4c None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:02.948175 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83407ba-c7be-43b2-a938-22c49e761a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:02.948446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83407ba-c7be-43b2-a938-22c49e761a4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:02.978752 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d305c-6f3e-4fc6-8c55-675ddded9f1a None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:02.991461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:03.016292 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494d2c7-a129-4e49-b3e0-6fbc962212ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:03.016896 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494d2c7-a129-4e49-b3e0-6fbc962212ec None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:03.017174 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494d2c7-a129-4e49-b3e0-6fbc962212ec None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:03.017457 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494d2c7-a129-4e49-b3e0-6fbc962212ec None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:03.018113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6494d2c7-a129-4e49-b3e0-6fbc962212ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:03.018468 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6494d2c7-a129-4e49-b3e0-6fbc962212ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:03.019890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6494d2c7-a129-4e49-b3e0-6fbc962212ec None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:03.033087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 256/527] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:03.033911 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494d2c7-a129-4e49-b3e0-6fbc962212ec None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:03.052283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83407ba-c7be-43b2-a938-22c49e761a4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:03.053758 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83407ba-c7be-43b2-a938-22c49e761a4c None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:03.115853 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:03.146579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494d2c7-a129-4e49-b3e0-6fbc962212ec None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:03.166101 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 272/528] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:32:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:32:03.166101 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83407ba-c7be-43b2-a938-22c49e761a4c None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:03.190547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d890348-5ea1-4caf-a4f9-2f963ffc496c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:03.191171 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d890348-5ea1-4caf-a4f9-2f963ffc496c None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:03.191452 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d890348-5ea1-4caf-a4f9-2f963ffc496c None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:03.191714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d890348-5ea1-4caf-a4f9-2f963ffc496c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:03.192438 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d890348-5ea1-4caf-a4f9-2f963ffc496c None admin] RBAC: Adding query filter params (role.id=7764573472d34ecc8bedf7a9c7935170, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=9daa7489bced467984d39e8b8ac49d13) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:03.192743 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d890348-5ea1-4caf-a4f9-2f963ffc496c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:03.194200 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d890348-5ea1-4caf-a4f9-2f963ffc496c None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:03.200498 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 257/529] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:32:03 2022] GET /identity/v3/role_assignments?user.id=9daa7489bced467984d39e8b8ac49d13&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=7764573472d34ecc8bedf7a9c7935170 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:03.201303 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d890348-5ea1-4caf-a4f9-2f963ffc496c None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:03.247865 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:03.288021 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83407ba-c7be-43b2-a938-22c49e761a4c None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:03.330692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712e6cc-c45e-4fb1-a596-8a135fbd00f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:03.331724 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712e6cc-c45e-4fb1-a596-8a135fbd00f3 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:03.332226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712e6cc-c45e-4fb1-a596-8a135fbd00f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:03.332707 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712e6cc-c45e-4fb1-a596-8a135fbd00f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:03.333590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712e6cc-c45e-4fb1-a596-8a135fbd00f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:03.334104 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712e6cc-c45e-4fb1-a596-8a135fbd00f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:03.335754 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712e6cc-c45e-4fb1-a596-8a135fbd00f3 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:03.336332 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d890348-5ea1-4caf-a4f9-2f963ffc496c None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:03.350671 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 273/530] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:03.352122 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712e6cc-c45e-4fb1-a596-8a135fbd00f3 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:03.378206 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67855e71-3865-45ff-b3d7-f90d565b5076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:03.379134 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67855e71-3865-45ff-b3d7-f90d565b5076 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:03.379246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67855e71-3865-45ff-b3d7-f90d565b5076 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:03.379508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67855e71-3865-45ff-b3d7-f90d565b5076 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:03.383094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:03.388576 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67855e71-3865-45ff-b3d7-f90d565b5076 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:03.390006 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67855e71-3865-45ff-b3d7-f90d565b5076 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:03.402553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-67855e71-3865-45ff-b3d7-f90d565b5076 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:03.402553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:03.402553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:03.402553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:03.402553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:03.402553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:03.402553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:03.402553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:03.402553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:03.402553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:03.402553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:03.402553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:03.402553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:03.402553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:03.402553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:03.403546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:03.403546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:03.403546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:03.403546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:03.403546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:03.403546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:03.403546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:03.403546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:03.403546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:03.403546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:03.403546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:03.403546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:03.403546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:03.403546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:03.403546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:03.403546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:03.403546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:03.404617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:03.404617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:03.404617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:03.404617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:03.404617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:03.404617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:03.404617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:03.404617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:03.404617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 258/531] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:03.405286 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67855e71-3865-45ff-b3d7-f90d565b5076 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:03.492082 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712e6cc-c45e-4fb1-a596-8a135fbd00f3 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:03.530664 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:03.541514 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20993a-4bd4-4a0b-a20d-09e2e81ccbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:03.543003 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20993a-4bd4-4a0b-a20d-09e2e81ccbeb None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:03.546419 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20993a-4bd4-4a0b-a20d-09e2e81ccbeb None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:03.546419 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20993a-4bd4-4a0b-a20d-09e2e81ccbeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:03.565183 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20993a-4bd4-4a0b-a20d-09e2e81ccbeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:03.568098 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20993a-4bd4-4a0b-a20d-09e2e81ccbeb None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:03.584415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67855e71-3865-45ff-b3d7-f90d565b5076 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:03.589307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-fe20993a-4bd4-4a0b-a20d-09e2e81ccbeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:03.589307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:03.589307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:03.589307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:03.589307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:03.589307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:03.589307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:03.589307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:03.589307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:03.589307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:03.589307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:03.589307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:03.589307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:03.589307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:03.589307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:03.589307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:03.590246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:03.590246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:03.590246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:03.590246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:03.590246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:03.590246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:03.590246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:03.590246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:03.590246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:03.590246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:03.590246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:03.590246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:03.590246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:03.590246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:03.590246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:03.590246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:03.590246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:03.591136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:03.591136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:03.591136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:03.594106 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 274/532] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:03.595743 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20993a-4bd4-4a0b-a20d-09e2e81ccbeb None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:03.627514 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867851d6-76a2-4fa4-a48b-1ef5fe4dfcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:03.628212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867851d6-76a2-4fa4-a48b-1ef5fe4dfcfe None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:03.628434 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867851d6-76a2-4fa4-a48b-1ef5fe4dfcfe None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:03.628705 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867851d6-76a2-4fa4-a48b-1ef5fe4dfcfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:03.638610 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867851d6-76a2-4fa4-a48b-1ef5fe4dfcfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:03.640839 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867851d6-76a2-4fa4-a48b-1ef5fe4dfcfe None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:03.657394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-867851d6-76a2-4fa4-a48b-1ef5fe4dfcfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:03.657394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:03.657394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:03.657394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:03.657394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:03.657394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:03.657394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:03.657394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:03.657394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:03.657394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:03.657394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:03.657394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:03.657394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:03.657394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:03.657394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:03.658348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:03.658348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:03.658348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:03.658348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:03.658348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:03.658348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:03.658348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:03.658348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:03.658348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:03.658348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:03.658348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:03.658348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:03.658348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:03.658348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:03.658348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:03.658348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:03.658348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:03.659248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:03.659248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:03.659248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:03.659248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:03.659248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:03.659248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:03.659248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:03.659248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:03.662708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 259/533] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:03.663799 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867851d6-76a2-4fa4-a48b-1ef5fe4dfcfe None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:03.734106 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20993a-4bd4-4a0b-a20d-09e2e81ccbeb None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:03.741752 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:03.780149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a72f1a-dbca-46c1-ade5-988f7507483f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:03.780829 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a72f1a-dbca-46c1-ade5-988f7507483f None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:03.781049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a72f1a-dbca-46c1-ade5-988f7507483f None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:03.781320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a72f1a-dbca-46c1-ade5-988f7507483f None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:03.782612 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a72f1a-dbca-46c1-ade5-988f7507483f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:03.782924 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a72f1a-dbca-46c1-ade5-988f7507483f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:03.785498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a72f1a-dbca-46c1-ade5-988f7507483f None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:03.794555 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867851d6-76a2-4fa4-a48b-1ef5fe4dfcfe None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:03.801618 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 275/534] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:03.802793 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a72f1a-dbca-46c1-ade5-988f7507483f None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:03.840218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492954c-1ab8-4db5-bc0e-ee2ff3c79bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:03.840829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492954c-1ab8-4db5-bc0e-ee2ff3c79bfc None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:03.841107 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492954c-1ab8-4db5-bc0e-ee2ff3c79bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:03.841366 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492954c-1ab8-4db5-bc0e-ee2ff3c79bfc None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:03.842186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492954c-1ab8-4db5-bc0e-ee2ff3c79bfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:03.842775 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492954c-1ab8-4db5-bc0e-ee2ff3c79bfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:03.844999 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492954c-1ab8-4db5-bc0e-ee2ff3c79bfc None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:03.851430 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:03.862437 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 260/535] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:03.865654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492954c-1ab8-4db5-bc0e-ee2ff3c79bfc None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:03.937547 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a72f1a-dbca-46c1-ade5-988f7507483f None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:03.976509 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0faee8-aef7-401e-bd30-82120dd180bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:03.977430 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0faee8-aef7-401e-bd30-82120dd180bb None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:03.978648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0faee8-aef7-401e-bd30-82120dd180bb None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:03.979381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0faee8-aef7-401e-bd30-82120dd180bb None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:03.980377 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0faee8-aef7-401e-bd30-82120dd180bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:03.980857 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0faee8-aef7-401e-bd30-82120dd180bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:03.982498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0faee8-aef7-401e-bd30-82120dd180bb None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:03.996147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 276/536] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:03.997259 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0faee8-aef7-401e-bd30-82120dd180bb None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:03.999946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492954c-1ab8-4db5-bc0e-ee2ff3c79bfc None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:04.021148 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:04.045004 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95b099-190f-4a7f-987b-71f30a562f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:04.045531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95b099-190f-4a7f-987b-71f30a562f21 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:04.045809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95b099-190f-4a7f-987b-71f30a562f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:04.046079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95b099-190f-4a7f-987b-71f30a562f21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:04.047153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95b099-190f-4a7f-987b-71f30a562f21 None admin] RBAC: Adding query filter params (role.id=7764573472d34ecc8bedf7a9c7935170, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=9482bf89c1b648c1a91ee00ee5fbba63) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:04.048553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95b099-190f-4a7f-987b-71f30a562f21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:04.050164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95b099-190f-4a7f-987b-71f30a562f21 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:04.057910 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 261/537] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:32:03 2022] GET /identity/v3/role_assignments?user.id=9482bf89c1b648c1a91ee00ee5fbba63&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=7764573472d34ecc8bedf7a9c7935170 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:04.058921 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95b099-190f-4a7f-987b-71f30a562f21 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:04.127296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:04.131045 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0faee8-aef7-401e-bd30-82120dd180bb None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:04.176359 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1d46a-2b9b-4325-9b09-cc514077ada9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:04.179150 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1d46a-2b9b-4325-9b09-cc514077ada9 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:04.179150 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1d46a-2b9b-4325-9b09-cc514077ada9 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:04.179150 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1d46a-2b9b-4325-9b09-cc514077ada9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:04.188110 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c1d46a-2b9b-4325-9b09-cc514077ada9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:04.190059 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c1d46a-2b9b-4325-9b09-cc514077ada9 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:04.204043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-51c1d46a-2b9b-4325-9b09-cc514077ada9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:04.204043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:04.204043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:04.204043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:04.204043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:04.204043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:04.204043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:04.204043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:04.204043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:04.204043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:04.204043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:04.204043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:04.204043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:04.204043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:04.204043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:04.205136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:04.205136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:04.205136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:04.205136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:04.205136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:04.205136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:04.205136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:04.205136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:04.205136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:04.205136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:04.205136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:04.205136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:04.205136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:04.205136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:04.205136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:04.205136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:04.205136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:04.206199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:04.206199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:04.206199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:04.206199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:04.206199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:04.206199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:04.206199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:04.206199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:04.213313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 277/538] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:04.215661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1d46a-2b9b-4325-9b09-cc514077ada9 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:04.230867 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95b099-190f-4a7f-987b-71f30a562f21 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:04.311443 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563e0b1-8ed2-49f5-8991-8c8d581e9ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:04.311994 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563e0b1-8ed2-49f5-8991-8c8d581e9ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:04.312279 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563e0b1-8ed2-49f5-8991-8c8d581e9ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:04.312556 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563e0b1-8ed2-49f5-8991-8c8d581e9ca5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:04.313311 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1563e0b1-8ed2-49f5-8991-8c8d581e9ca5 None admin] RBAC: Adding query filter params (role.id=7764573472d34ecc8bedf7a9c7935170, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=ea51ef02128a4b63b11049eb0daef298) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:04.313619 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1563e0b1-8ed2-49f5-8991-8c8d581e9ca5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:04.315119 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1563e0b1-8ed2-49f5-8991-8c8d581e9ca5 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:04.336653 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 262/539] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:32:04 2022] GET /identity/v3/role_assignments?user.id=ea51ef02128a4b63b11049eb0daef298&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=7764573472d34ecc8bedf7a9c7935170 => generated 259 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:04.338412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8da8a-4632-4f24-bb7c-3b3bc51b1fc8 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:04.338730 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8da8a-4632-4f24-bb7c-3b3bc51b1fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:04.339003 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8da8a-4632-4f24-bb7c-3b3bc51b1fc8 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:04.340412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 263/540] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:04.346361 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdf5d5-369d-4b21-89ff-95f72cfa3412 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:04.346998 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdf5d5-369d-4b21-89ff-95f72cfa3412 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:04.347475 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdf5d5-369d-4b21-89ff-95f72cfa3412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:04.367970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-f7fdf5d5-369d-4b21-89ff-95f72cfa3412 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:04.370000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-f7fdf5d5-369d-4b21-89ff-95f72cfa3412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:04.384814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1d46a-2b9b-4325-9b09-cc514077ada9 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:04.447215 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:04.470461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d760ff7-4005-440e-81f5-f15eb77c3345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:04.471208 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d760ff7-4005-440e-81f5-f15eb77c3345 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:04.471486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d760ff7-4005-440e-81f5-f15eb77c3345 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:04.471755 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d760ff7-4005-440e-81f5-f15eb77c3345 None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:04.472466 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d760ff7-4005-440e-81f5-f15eb77c3345 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:04.472831 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d760ff7-4005-440e-81f5-f15eb77c3345 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:04.474308 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d760ff7-4005-440e-81f5-f15eb77c3345 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:04.503563 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 278/541] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:32:04.505387 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 264/542] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:32:04.513296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d760ff7-4005-440e-81f5-f15eb77c3345 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:04.546677 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:04.559864 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdf5d5-369d-4b21-89ff-95f72cfa3412 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:04.666560 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d760ff7-4005-440e-81f5-f15eb77c3345 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:04.675492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdf5d5-369d-4b21-89ff-95f72cfa3412 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:04.702263 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eebaca-3363-415c-804f-512f019eec35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:04.702912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eebaca-3363-415c-804f-512f019eec35 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:04.703186 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eebaca-3363-415c-804f-512f019eec35 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:04.703451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eebaca-3363-415c-804f-512f019eec35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:04.704184 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eebaca-3363-415c-804f-512f019eec35 None admin] RBAC: Adding query filter params (role.id=7764573472d34ecc8bedf7a9c7935170, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=7bbf0678f34e46e8b56aaeabce955180) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:04.704488 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eebaca-3363-415c-804f-512f019eec35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:04.705921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eebaca-3363-415c-804f-512f019eec35 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:04.718609 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 279/543] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:32:04 2022] GET /identity/v3/role_assignments?user.id=7bbf0678f34e46e8b56aaeabce955180&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=7764573472d34ecc8bedf7a9c7935170 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:04.729028 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c92f1-e138-4896-89fa-8457e4fece9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:04.730082 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c92f1-e138-4896-89fa-8457e4fece9b None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:04.730561 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c92f1-e138-4896-89fa-8457e4fece9b None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:04.731047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c92f1-e138-4896-89fa-8457e4fece9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:04.747606 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c92f1-e138-4896-89fa-8457e4fece9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:32:04.753920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-1b5c92f1-e138-4896-89fa-8457e4fece9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:04.753920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:04.753920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:04.753920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:04.753920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:04.753920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:04.753920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:04.753920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:04.753920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:04.753920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:04.753920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:04.753920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:04.753920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:32:04.753920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:32:04.753920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:32:04.753920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:32:04.754890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:32:04.754890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:32:04.754890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:32:04.754890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:32:04.754890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:04.754890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:04.754890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:04.754890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:04.754890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:04.754890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:04.754890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:04.754890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:04.754890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:04.754890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:04.754890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:04.754890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:04.755723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:04.755723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:04.755723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:04.755723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:04.755723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:32:04.755723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:32:04.755723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:32:04.755723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:32:04.755723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:32:04.755723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:32:04.755723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:04.755723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:04.758650 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 265/544] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:32:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:04.764821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eebaca-3363-415c-804f-512f019eec35 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:04.890057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:04.907530 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eebaca-3363-415c-804f-512f019eec35 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:04.958289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5045c-6ad7-4dd9-804a-2ba65e1af003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:04.958945 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5045c-6ad7-4dd9-804a-2ba65e1af003 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:04.959218 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5045c-6ad7-4dd9-804a-2ba65e1af003 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:04.959489 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5045c-6ad7-4dd9-804a-2ba65e1af003 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:04.960154 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd5045c-6ad7-4dd9-804a-2ba65e1af003 None admin] RBAC: Adding query filter params (name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:04.960450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd5045c-6ad7-4dd9-804a-2ba65e1af003 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:04.961817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd5045c-6ad7-4dd9-804a-2ba65e1af003 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:04.972578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 280/545] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Aug 8 16:32:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:04.983625 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c92f1-e138-4896-89fa-8457e4fece9b None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:05.030501 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:05.114150 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c92f1-e138-4896-89fa-8457e4fece9b None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:05.154072 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17420881-0caf-496d-b5f2-0596df8118d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:05.154740 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17420881-0caf-496d-b5f2-0596df8118d1 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:05.155021 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17420881-0caf-496d-b5f2-0596df8118d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:05.155292 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17420881-0caf-496d-b5f2-0596df8118d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:05.169810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17420881-0caf-496d-b5f2-0596df8118d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:05.171318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17420881-0caf-496d-b5f2-0596df8118d1 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:05.189865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-17420881-0caf-496d-b5f2-0596df8118d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:05.189865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:05.189865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:05.189865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:05.189865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:05.189865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:05.189865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:05.189865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:05.189865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:05.189865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:05.189865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:05.189865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:05.189865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:05.189865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:05.189865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:05.189865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:05.190847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:05.190847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:05.190847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:05.190847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:05.190847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:05.190847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:05.190847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:05.190847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:05.190847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:05.190847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:05.190847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:05.190847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:05.190847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:05.190847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:05.190847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:05.190847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:05.190847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:05.191779 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:05.191779 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:05.191779 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:05.195152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 266/546] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:05.200844 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5045c-6ad7-4dd9-804a-2ba65e1af003 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:05.370216 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5045c-6ad7-4dd9-804a-2ba65e1af003 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:05.414538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cffae4-aa26-4b44-8f0a-32baf80cac9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:05.415204 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cffae4-aa26-4b44-8f0a-32baf80cac9a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:05.416036 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cffae4-aa26-4b44-8f0a-32baf80cac9a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:05.416482 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cffae4-aa26-4b44-8f0a-32baf80cac9a None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:05.417346 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cffae4-aa26-4b44-8f0a-32baf80cac9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:05.417643 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cffae4-aa26-4b44-8f0a-32baf80cac9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:05.419044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cffae4-aa26-4b44-8f0a-32baf80cac9a None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:05.433718 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 281/547] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:05.439095 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17420881-0caf-496d-b5f2-0596df8118d1 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:05.470360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7272e-0792-4a83-a6ca-006ef5f0f6be None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:05.470360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7272e-0792-4a83-a6ca-006ef5f0f6be None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:05.470360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7272e-0792-4a83-a6ca-006ef5f0f6be None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:05.470360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 282/548] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:05.476501 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8c5b9-b09f-4332-a476-6aa7413a5727 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:05.476807 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8c5b9-b09f-4332-a476-6aa7413a5727 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:05.477062 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8c5b9-b09f-4332-a476-6aa7413a5727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:05.515398 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:05.517717 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-7ac8c5b9-b09f-4332-a476-6aa7413a5727 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:05.519464 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-7ac8c5b9-b09f-4332-a476-6aa7413a5727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:05.608235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:05.640588 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17420881-0caf-496d-b5f2-0596df8118d1 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:05.661547 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 283/549] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:32:05.680175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee20677-8d76-4a8d-ae34-f6cbf52e45a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:05.681382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee20677-8d76-4a8d-ae34-f6cbf52e45a4 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:05.681640 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee20677-8d76-4a8d-ae34-f6cbf52e45a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:05.681882 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee20677-8d76-4a8d-ae34-f6cbf52e45a4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:05.691055 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee20677-8d76-4a8d-ae34-f6cbf52e45a4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:05.691977 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8c5b9-b09f-4332-a476-6aa7413a5727 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:05.692522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee20677-8d76-4a8d-ae34-f6cbf52e45a4 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:05.710609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-bee20677-8d76-4a8d-ae34-f6cbf52e45a4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:32:05.710609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:05.710609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:05.710609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:05.710609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:05.710609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:05.710609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:05.710609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:05.710609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:05.710609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:05.710609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:05.710609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:05.710609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:05.710609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:05.710609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:05.710609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:05.711614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:05.711614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:05.711614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:05.711614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:05.711614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:05.711614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:05.711614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:05.711614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:05.711614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:05.711614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:05.711614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:05.711614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:05.711614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:05.711614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:05.711614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:05.711614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:05.711614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:05.711614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:05.712572 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:05.712572 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:05.712572 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:05.712572 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:05.712572 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:05.712572 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:05.712572 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:05.712572 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:05.712572 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:32:05.712572 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:05.712572 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 267/550] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:32:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:05.713273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee20677-8d76-4a8d-ae34-f6cbf52e45a4 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:05.835978 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8c5b9-b09f-4332-a476-6aa7413a5727 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:05.920830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1e35f-376a-4023-a88e-f172ab14f171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:05.921939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1e35f-376a-4023-a88e-f172ab14f171 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:05.923918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1e35f-376a-4023-a88e-f172ab14f171 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:05.924478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1e35f-376a-4023-a88e-f172ab14f171 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:05.932626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:05.944338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee20677-8d76-4a8d-ae34-f6cbf52e45a4 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:05.957010 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f1e35f-376a-4023-a88e-f172ab14f171 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:32:05.967678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-76f1e35f-376a-4023-a88e-f172ab14f171 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:05.967678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:05.967678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:05.967678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:05.967678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:05.967678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:05.967678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:05.967678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:05.967678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:05.967678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:05.967678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:05.967678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:05.967678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:32:05.967678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:32:05.967678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:32:05.967678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:32:05.968847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:32:05.968847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:32:05.968847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:32:05.968847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:32:05.968847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:05.968847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:05.968847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:05.968847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:05.968847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:05.968847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:05.968847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:05.968847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:05.968847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:05.968847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:05.968847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:05.968847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:05.969692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:05.969692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:05.969692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:05.969692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:05.969692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:32:05.969692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:32:05.969692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:32:05.969692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:32:05.969692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:32:05.969692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:32:05.969692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:05.969692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:05.969692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 284/551] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:32:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:32:05.975276 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1e35f-376a-4023-a88e-f172ab14f171 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:06.018982 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f209ef8-d4eb-40cb-9cd8-67ce6ff35d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:06.019591 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f209ef8-d4eb-40cb-9cd8-67ce6ff35d5f None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:06.019866 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f209ef8-d4eb-40cb-9cd8-67ce6ff35d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:06.020131 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f209ef8-d4eb-40cb-9cd8-67ce6ff35d5f None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:06.020856 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f209ef8-d4eb-40cb-9cd8-67ce6ff35d5f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:06.021156 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f209ef8-d4eb-40cb-9cd8-67ce6ff35d5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:06.022609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f209ef8-d4eb-40cb-9cd8-67ce6ff35d5f None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:06.096420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 268/552] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Aug 8 16:32:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:06.102254 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f209ef8-d4eb-40cb-9cd8-67ce6ff35d5f None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:06.115267 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:06.191750 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1e35f-376a-4023-a88e-f172ab14f171 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:06.255329 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f209ef8-d4eb-40cb-9cd8-67ce6ff35d5f None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:06.257500 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3b7b1-714d-4fb4-9a2f-da40176cc363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:06.258120 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3b7b1-714d-4fb4-9a2f-da40176cc363 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:06.258396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3b7b1-714d-4fb4-9a2f-da40176cc363 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:06.258692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3b7b1-714d-4fb4-9a2f-da40176cc363 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:06.259427 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3b7b1-714d-4fb4-9a2f-da40176cc363 None admin] RBAC: Adding query filter params (name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:06.259726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3b7b1-714d-4fb4-9a2f-da40176cc363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:06.262097 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3b7b1-714d-4fb4-9a2f-da40176cc363 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:06.279728 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 285/553] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Aug 8 16:32:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:06.287705 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3b7b1-714d-4fb4-9a2f-da40176cc363 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:06.331043 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d623c75-0c57-4a11-9c09-1ea7590295ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:06.331705 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d623c75-0c57-4a11-9c09-1ea7590295ad None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:06.331988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d623c75-0c57-4a11-9c09-1ea7590295ad None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:06.332260 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d623c75-0c57-4a11-9c09-1ea7590295ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:06.390640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:06.467691 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d623c75-0c57-4a11-9c09-1ea7590295ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:06.470486 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d623c75-0c57-4a11-9c09-1ea7590295ad None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:06.475745 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:06.492707 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3b7b1-714d-4fb4-9a2f-da40176cc363 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:06.554581 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33662278-7811-4718-a9a3-4c4f524f1a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:06.555686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33662278-7811-4718-a9a3-4c4f524f1a11 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:06.556707 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33662278-7811-4718-a9a3-4c4f524f1a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:06.557187 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33662278-7811-4718-a9a3-4c4f524f1a11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:06.577109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33662278-7811-4718-a9a3-4c4f524f1a11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:06.579861 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33662278-7811-4718-a9a3-4c4f524f1a11 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:06.625974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-33662278-7811-4718-a9a3-4c4f524f1a11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:06.625974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:06.625974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:06.625974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:06.625974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:06.625974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:06.625974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:06.625974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:06.625974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:06.625974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:06.625974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:06.625974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:06.625974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:06.625974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:06.625974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:06.625974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:06.627083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:06.627083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:06.627083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:06.627083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:06.627083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:06.627083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:06.627083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:06.627083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:06.627083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:06.627083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:06.627083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:06.627083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:06.627083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:06.627083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:06.627083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:06.627083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:06.627083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:06.627991 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:06.627991 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:06.627991 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:06.631598 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 286/554] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:06.635509 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765c270-b4f4-43ec-8b53-f0c1644db21d None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:06.635790 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765c270-b4f4-43ec-8b53-f0c1644db21d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:06.636089 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765c270-b4f4-43ec-8b53-f0c1644db21d None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:06.637333 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 287/555] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:32:06.644397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd7d79-21d8-4f93-8654-d49b0823c319 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:06.644630 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd7d79-21d8-4f93-8654-d49b0823c319 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:06.644870 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd7d79-21d8-4f93-8654-d49b0823c319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:06.667105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-4ecd7d79-21d8-4f93-8654-d49b0823c319 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:06.669206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-4ecd7d79-21d8-4f93-8654-d49b0823c319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:06.766558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 288/556] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:32:06.767879 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd7d79-21d8-4f93-8654-d49b0823c319 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:06.778257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 269/557] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:32:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 676 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:32:06.785023 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d623c75-0c57-4a11-9c09-1ea7590295ad None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:06.822188 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:06.891510 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd7d79-21d8-4f93-8654-d49b0823c319 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:06.909011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d623c75-0c57-4a11-9c09-1ea7590295ad None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:06.912878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:06.940832 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690c1c7-3953-4efa-bd7c-7000ddd6eae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:06.941542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690c1c7-3953-4efa-bd7c-7000ddd6eae6 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:06.941893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690c1c7-3953-4efa-bd7c-7000ddd6eae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:06.942169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690c1c7-3953-4efa-bd7c-7000ddd6eae6 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:06.942868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690c1c7-3953-4efa-bd7c-7000ddd6eae6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:06.943162 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690c1c7-3953-4efa-bd7c-7000ddd6eae6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:06.944541 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690c1c7-3953-4efa-bd7c-7000ddd6eae6 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:06.952460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0415b59-a825-42db-9c79-c9c422e8822f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:06.952872 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0415b59-a825-42db-9c79-c9c422e8822f None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:06.953145 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0415b59-a825-42db-9c79-c9c422e8822f None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:06.953410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0415b59-a825-42db-9c79-c9c422e8822f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:06.957831 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 289/558] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:06.958520 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690c1c7-3953-4efa-bd7c-7000ddd6eae6 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:06.969716 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0415b59-a825-42db-9c79-c9c422e8822f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:06.971271 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0415b59-a825-42db-9c79-c9c422e8822f None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:06.988645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-a0415b59-a825-42db-9c79-c9c422e8822f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:06.988645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:06.988645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:06.988645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:06.988645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:06.988645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:06.988645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:06.988645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:06.988645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:06.988645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:06.988645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:06.988645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:06.988645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:06.988645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:06.988645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:06.988645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:06.990210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:06.990210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:06.990210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:06.990210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:06.990210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:06.990210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:06.990210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:06.990210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:06.990210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:06.990210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:06.990210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:06.990210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:06.990210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:06.990210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:06.990210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:06.990210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:06.990210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:06.992129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:06.992129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:06.992129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:06.992129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 270/559] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:06.992129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b7a16-82d5-4973-9991-97effe0a4791 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:06.992129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b7a16-82d5-4973-9991-97effe0a4791 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:06.992129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b7a16-82d5-4973-9991-97effe0a4791 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:06.998113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 271/560] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:06.998113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b7a16-82d5-4973-9991-97effe0a4791 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:07.073087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:07.076599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690c1c7-3953-4efa-bd7c-7000ddd6eae6 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:07.141672 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b7a16-82d5-4973-9991-97effe0a4791 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:07.141930 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7cb5e-c22f-4bf8-90f9-358532c035c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:07.141930 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7cb5e-c22f-4bf8-90f9-358532c035c0 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:07.141930 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7cb5e-c22f-4bf8-90f9-358532c035c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:07.141930 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7cb5e-c22f-4bf8-90f9-358532c035c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:07.142292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7cb5e-c22f-4bf8-90f9-358532c035c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:32:07.142292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-9be7cb5e-c22f-4bf8-90f9-358532c035c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:07.142292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:07.142292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:07.142292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:07.142292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:07.142292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:07.142292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:07.142292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:07.142292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:07.142292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:07.142292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:07.142292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:07.142292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:32:07.143152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:32:07.143152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:32:07.143152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:32:07.143152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:32:07.143152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:32:07.143152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:32:07.143152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:32:07.143152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:07.143152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:07.143152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:07.143152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:07.143152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:07.143152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:07.143152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:07.143152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:07.143152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:07.143152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:07.144350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:07.144350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:07.144350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:07.144350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:07.144350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:07.144350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:07.144350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:32:07.144350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:32:07.144350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:32:07.144350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:32:07.144350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:32:07.144350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:32:07.144350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:07.144350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:07.144350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 290/561] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:32:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:07.145348 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7cb5e-c22f-4bf8-90f9-358532c035c0 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:07.162677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744317d-dcc4-4a30-ab40-816b0f520cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:07.162677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744317d-dcc4-4a30-ab40-816b0f520cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:07.162677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744317d-dcc4-4a30-ab40-816b0f520cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:07.163010 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744317d-dcc4-4a30-ab40-816b0f520cf7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:07.183660 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744317d-dcc4-4a30-ab40-816b0f520cf7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:07.186099 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744317d-dcc4-4a30-ab40-816b0f520cf7 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:07.200141 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-7744317d-dcc4-4a30-ab40-816b0f520cf7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:32:07.200141 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:07.200141 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:07.200141 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:07.200141 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:07.200141 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:07.200141 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:07.200141 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:07.200141 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:07.200141 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:07.200141 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:07.200141 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:07.200141 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:07.200141 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:07.200141 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:07.200141 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:07.201186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:07.201186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:07.201186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:07.201186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:07.201186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:07.201186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:07.201186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:07.201186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:07.201186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:07.201186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:07.201186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:07.201186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:07.201186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:07.201186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:07.201186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:07.201186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:07.201186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:07.201186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:07.202122 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:07.202122 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:07.202122 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:07.202122 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:07.202122 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:07.202122 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:07.202122 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:07.202122 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:07.202122 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:32:07.202122 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:07.202122 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 272/562] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Aug 8 16:32:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:07.202860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de201fd0-1b2c-4915-b8af-280091249101 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:07.202860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de201fd0-1b2c-4915-b8af-280091249101 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:07.202983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de201fd0-1b2c-4915-b8af-280091249101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:07.222097 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-de201fd0-1b2c-4915-b8af-280091249101 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:07.223326 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-de201fd0-1b2c-4915-b8af-280091249101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:07.236639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:07.266301 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7cb5e-c22f-4bf8-90f9-358532c035c0 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:07.298684 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 273/563] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:32:07.299977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de201fd0-1b2c-4915-b8af-280091249101 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:07.309772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e55c3-d3c7-46aa-b985-330b81f97f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:07.310440 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e55c3-d3c7-46aa-b985-330b81f97f1d None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:07.310748 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e55c3-d3c7-46aa-b985-330b81f97f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:07.311019 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e55c3-d3c7-46aa-b985-330b81f97f1d None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:07.311669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e55c3-d3c7-46aa-b985-330b81f97f1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:07.311965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e55c3-d3c7-46aa-b985-330b81f97f1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:07.313345 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e55c3-d3c7-46aa-b985-330b81f97f1d None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:07.331347 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 291/564] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:07.332276 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e55c3-d3c7-46aa-b985-330b81f97f1d None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:07.357286 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:07.428376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de201fd0-1b2c-4915-b8af-280091249101 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:07.458502 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e55c3-d3c7-46aa-b985-330b81f97f1d None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:07.475595 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb910d7-9640-45e2-abb8-fcf364a6ef67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:07.476400 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb910d7-9640-45e2-abb8-fcf364a6ef67 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:07.476508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb910d7-9640-45e2-abb8-fcf364a6ef67 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:07.476728 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb910d7-9640-45e2-abb8-fcf364a6ef67 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:07.477398 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb910d7-9640-45e2-abb8-fcf364a6ef67 None admin] RBAC: Adding query filter params (name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:07.477772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb910d7-9640-45e2-abb8-fcf364a6ef67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:07.479950 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb910d7-9640-45e2-abb8-fcf364a6ef67 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:07.489853 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 274/565] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Aug 8 16:32:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:07.490911 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb910d7-9640-45e2-abb8-fcf364a6ef67 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:07.504424 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d4224-a9e5-4549-b309-41a2b0b3bb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:07.505043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d4224-a9e5-4549-b309-41a2b0b3bb56 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:07.505319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d4224-a9e5-4549-b309-41a2b0b3bb56 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:07.505450 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:07.505591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d4224-a9e5-4549-b309-41a2b0b3bb56 None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:07.507071 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d4224-a9e5-4549-b309-41a2b0b3bb56 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:07.507373 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d4224-a9e5-4549-b309-41a2b0b3bb56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:07.508851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d4224-a9e5-4549-b309-41a2b0b3bb56 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:07.529934 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 292/566] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Aug 8 16:32:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:07.530739 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d4224-a9e5-4549-b309-41a2b0b3bb56 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:07.609232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb910d7-9640-45e2-abb8-fcf364a6ef67 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:07.617928 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:07.640126 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d4224-a9e5-4549-b309-41a2b0b3bb56 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:07.648682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7b4fa-ce85-4e18-9cff-214ee221d628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:07.649319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7b4fa-ce85-4e18-9cff-214ee221d628 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:07.650198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7b4fa-ce85-4e18-9cff-214ee221d628 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:07.651034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7b4fa-ce85-4e18-9cff-214ee221d628 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:07.661833 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7b4fa-ce85-4e18-9cff-214ee221d628 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:32:07.667152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-8de7b4fa-ce85-4e18-9cff-214ee221d628 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:07.667152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:07.667152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:07.667152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:07.667152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:07.667152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:07.667152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:07.667152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:07.667152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:07.667152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:07.667152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:07.667152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:07.667152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:32:07.667152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:32:07.667152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:32:07.667152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:32:07.668125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:32:07.668125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:32:07.668125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:32:07.668125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:32:07.668125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:07.668125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:07.668125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:07.668125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:07.668125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:07.668125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:07.668125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:07.668125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:07.668125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:07.668125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:07.668125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:07.668125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:07.668970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:07.668970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:07.668970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:07.668970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:07.668970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:32:07.668970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:32:07.668970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:32:07.668970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:32:07.668970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:32:07.668970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:32:07.668970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:07.668970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:07.668970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 275/567] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:32:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:07.673295 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2513d-905a-4f9b-84cf-03569675bb84 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:07.673295 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2513d-905a-4f9b-84cf-03569675bb84 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:07.673295 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2513d-905a-4f9b-84cf-03569675bb84 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:07.673295 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 276/568] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:07.673295 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2513d-905a-4f9b-84cf-03569675bb84 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:07.687166 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13004d56-e409-4e5c-992e-9964d902339a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:07.687699 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13004d56-e409-4e5c-992e-9964d902339a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:07.687969 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13004d56-e409-4e5c-992e-9964d902339a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:07.688233 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13004d56-e409-4e5c-992e-9964d902339a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:07.699238 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13004d56-e409-4e5c-992e-9964d902339a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:07.701962 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13004d56-e409-4e5c-992e-9964d902339a None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:07.721640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-13004d56-e409-4e5c-992e-9964d902339a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:07.721640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:07.721640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:07.721640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:07.721640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:07.721640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:07.721640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:07.721640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:07.721640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:07.721640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:07.721640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:07.721640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:07.721640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:07.721640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:07.721640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:07.722666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:07.722666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:07.722666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:07.722666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:07.722666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:07.722666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:07.722666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:07.722666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:07.722666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:07.722666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:07.722666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:07.722666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:07.722666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:07.722666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:07.722666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:07.722666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:07.722666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:07.723581 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:07.723581 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:07.723581 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:07.723581 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:07.723581 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:07.723581 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:07.723581 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:07.723581 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:07.723581 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 293/569] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:07.724613 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13004d56-e409-4e5c-992e-9964d902339a None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:07.771431 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:07.809114 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2513d-905a-4f9b-84cf-03569675bb84 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:07.848559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32bb2e1-658a-4c05-ab13-836e1a879712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:07.849361 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bb2e1-658a-4c05-ab13-836e1a879712 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:07.849700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bb2e1-658a-4c05-ab13-836e1a879712 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:07.850157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bb2e1-658a-4c05-ab13-836e1a879712 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:07.854287 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13004d56-e409-4e5c-992e-9964d902339a None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:07.863931 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32bb2e1-658a-4c05-ab13-836e1a879712 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:07.865407 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32bb2e1-658a-4c05-ab13-836e1a879712 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:07.882585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-c32bb2e1-658a-4c05-ab13-836e1a879712 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:07.882585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:07.882585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:07.882585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:07.882585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:07.882585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:07.882585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:07.882585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:07.882585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:07.882585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:07.882585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:07.882585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:07.882585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:07.882585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:07.882585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:07.882585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:07.883722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:07.883722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:07.883722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:07.883722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:07.883722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:07.883722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:07.883722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:07.883722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:07.883722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:07.883722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:07.883722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:07.883722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:07.883722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:07.883722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:07.883722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:07.883722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:07.883722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:07.885579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:07.885579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:07.885579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:07.889142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 277/570] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:07.890084 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32bb2e1-658a-4c05-ab13-836e1a879712 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:07.914346 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22173323-5dbe-4c03-bb40-59a24b2727a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:07.915068 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22173323-5dbe-4c03-bb40-59a24b2727a6 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:07.915939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22173323-5dbe-4c03-bb40-59a24b2727a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:07.916310 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22173323-5dbe-4c03-bb40-59a24b2727a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:07.925459 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:07.933906 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22173323-5dbe-4c03-bb40-59a24b2727a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:07.935435 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22173323-5dbe-4c03-bb40-59a24b2727a6 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:07.952065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-22173323-5dbe-4c03-bb40-59a24b2727a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:07.952065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:07.952065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:07.952065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:07.952065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:07.952065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:07.952065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:07.952065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:07.952065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:07.952065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:07.952065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:07.952065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:07.952065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:07.952065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:07.952065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:07.952065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:07.953077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:07.953077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:07.953077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:07.953077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:07.953077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:07.953077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:07.953077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:07.953077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:07.953077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:07.953077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:07.953077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:07.953077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:07.953077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:07.953077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:07.953077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:07.953077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:07.953077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:07.954213 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:07.954213 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:07.954213 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:07.958044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 294/571] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:07.959985 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4443bc1-5db9-4d0f-b8e1-8b4686408528 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:07.960460 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4443bc1-5db9-4d0f-b8e1-8b4686408528 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:07.962036 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4443bc1-5db9-4d0f-b8e1-8b4686408528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:07.988073 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-d4443bc1-5db9-4d0f-b8e1-8b4686408528 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:07.990087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-d4443bc1-5db9-4d0f-b8e1-8b4686408528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:08.023308 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32bb2e1-658a-4c05-ab13-836e1a879712 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:08.048497 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:08.067370 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e52613-d40d-4321-971c-d6fa247d1ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:08.067981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e52613-d40d-4321-971c-d6fa247d1ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:08.068251 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e52613-d40d-4321-971c-d6fa247d1ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:08.068518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e52613-d40d-4321-971c-d6fa247d1ed8 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:08.069181 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e52613-d40d-4321-971c-d6fa247d1ed8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:08.069479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e52613-d40d-4321-971c-d6fa247d1ed8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:08.070959 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e52613-d40d-4321-971c-d6fa247d1ed8 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:08.079079 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 295/572] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:32:08.079845 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4443bc1-5db9-4d0f-b8e1-8b4686408528 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:08.085893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 278/573] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Aug 8 16:32:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:08.085893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e52613-d40d-4321-971c-d6fa247d1ed8 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:08.208890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e52613-d40d-4321-971c-d6fa247d1ed8 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:08.221104 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4443bc1-5db9-4d0f-b8e1-8b4686408528 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:08.248502 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:08.262330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e43f82-ae2b-4a59-b525-862ffac25910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:08.263538 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e43f82-ae2b-4a59-b525-862ffac25910 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:08.264001 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e43f82-ae2b-4a59-b525-862ffac25910 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:08.264450 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e43f82-ae2b-4a59-b525-862ffac25910 None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:08.265432 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078478d-0d18-4795-a2ca-0573886f1adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:08.265618 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e43f82-ae2b-4a59-b525-862ffac25910 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:08.266170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e43f82-ae2b-4a59-b525-862ffac25910 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:08.266290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078478d-0d18-4795-a2ca-0573886f1adc None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:08.266491 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078478d-0d18-4795-a2ca-0573886f1adc None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:08.266799 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078478d-0d18-4795-a2ca-0573886f1adc None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:08.267489 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078478d-0d18-4795-a2ca-0573886f1adc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:08.267783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078478d-0d18-4795-a2ca-0573886f1adc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:08.268867 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e43f82-ae2b-4a59-b525-862ffac25910 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:08.269189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078478d-0d18-4795-a2ca-0573886f1adc None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:08.286005 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 279/574] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:08.286936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e43f82-ae2b-4a59-b525-862ffac25910 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:08.290007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 296/575] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:08.291250 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078478d-0d18-4795-a2ca-0573886f1adc None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:08.351009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:08.424120 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078478d-0d18-4795-a2ca-0573886f1adc None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:08.435163 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e43f82-ae2b-4a59-b525-862ffac25910 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:08.464229 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f898fac-0fda-4816-8602-a8c125e22265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:08.464781 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f898fac-0fda-4816-8602-a8c125e22265 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:08.465056 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f898fac-0fda-4816-8602-a8c125e22265 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:08.465320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f898fac-0fda-4816-8602-a8c125e22265 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:08.479364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f898fac-0fda-4816-8602-a8c125e22265 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:08.480863 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f898fac-0fda-4816-8602-a8c125e22265 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:08.486621 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d883425-bbb6-48d0-888f-72c03acf758b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:08.487601 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d883425-bbb6-48d0-888f-72c03acf758b None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:08.488206 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d883425-bbb6-48d0-888f-72c03acf758b None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:08.488592 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d883425-bbb6-48d0-888f-72c03acf758b None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:08.489346 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d883425-bbb6-48d0-888f-72c03acf758b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:08.489780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d883425-bbb6-48d0-888f-72c03acf758b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:08.491819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d883425-bbb6-48d0-888f-72c03acf758b None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:08.497782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-7f898fac-0fda-4816-8602-a8c125e22265 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:08.497782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:08.497782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:08.497782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:08.497782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:08.497782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:08.497782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:08.497782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:08.497782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:08.497782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:08.497782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:08.497782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:08.497782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:08.497782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:08.497782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:08.497782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:08.498903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:08.498903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:08.498903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:08.498903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:08.498903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:08.498903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:08.498903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:08.498903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:08.498903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:08.498903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:08.498903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:08.498903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:08.498903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:08.498903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:08.498903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:08.498903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:08.498903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:08.500121 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:08.500121 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:08.500121 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:08.500121 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 297/576] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:08.500362 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f898fac-0fda-4816-8602-a8c125e22265 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:08.513477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 280/577] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:08.514901 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d883425-bbb6-48d0-888f-72c03acf758b None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:08.546428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:08.638673 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f898fac-0fda-4816-8602-a8c125e22265 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:08.657063 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d883425-bbb6-48d0-888f-72c03acf758b None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:08.658872 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:08.689024 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c6247-3891-487b-9010-6532f6d7d5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:08.689974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c6247-3891-487b-9010-6532f6d7d5ae None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:08.690246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c6247-3891-487b-9010-6532f6d7d5ae None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:08.690501 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c6247-3891-487b-9010-6532f6d7d5ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:08.707164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12d77a-8138-4191-ad02-50a2d46abdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:08.707940 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12d77a-8138-4191-ad02-50a2d46abdee None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:08.708290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12d77a-8138-4191-ad02-50a2d46abdee None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:08.708724 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12d77a-8138-4191-ad02-50a2d46abdee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:08.709419 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12d77a-8138-4191-ad02-50a2d46abdee None admin] RBAC: Adding query filter params (role.id=7764573472d34ecc8bedf7a9c7935170, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=c606decb832444e38a194b98f8d15f8c) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:08.709810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12d77a-8138-4191-ad02-50a2d46abdee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:08.710715 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478c6247-3891-487b-9010-6532f6d7d5ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:32:08.711305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12d77a-8138-4191-ad02-50a2d46abdee None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:08.718755 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 281/578] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:32:08 2022] GET /identity/v3/role_assignments?user.id=c606decb832444e38a194b98f8d15f8c&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=7764573472d34ecc8bedf7a9c7935170 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:08.719534 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12d77a-8138-4191-ad02-50a2d46abdee None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:08.721968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-478c6247-3891-487b-9010-6532f6d7d5ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:08.721968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:08.721968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:08.721968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:08.721968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:08.721968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:08.721968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:08.721968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:08.721968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:08.721968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:08.721968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:08.721968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:08.721968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:32:08.721968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:32:08.721968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:32:08.721968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:32:08.723035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:32:08.723035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:32:08.723035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:32:08.723035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:32:08.723035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:08.723035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:08.723035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:08.723035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:08.723035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:08.723035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:08.723035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:08.723035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:08.723035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:08.723035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:08.723035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:08.723035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:08.723914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:08.723914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:08.723914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:08.723914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:08.723914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:32:08.723914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:32:08.723914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:32:08.723914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:32:08.723914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:32:08.723914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:32:08.723914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:08.723914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:08.734345 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 298/579] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:32:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:08.736476 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c6247-3891-487b-9010-6532f6d7d5ae None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:08.883542 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12d77a-8138-4191-ad02-50a2d46abdee None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:08.901402 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:08.916641 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c6247-3891-487b-9010-6532f6d7d5ae None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:08.954817 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a620482-9459-4c5e-9e91-b6a467009e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:08.955860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a620482-9459-4c5e-9e91-b6a467009e59 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:08.956342 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a620482-9459-4c5e-9e91-b6a467009e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:08.956814 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a620482-9459-4c5e-9e91-b6a467009e59 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:08.969202 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a620482-9459-4c5e-9e91-b6a467009e59 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:08.972087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a620482-9459-4c5e-9e91-b6a467009e59 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:08.991054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-5a620482-9459-4c5e-9e91-b6a467009e59 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 16:32:08.991054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:08.991054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:08.991054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:08.991054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:08.991054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:08.991054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:08.991054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:08.991054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:08.991054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:08.991054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:08.991054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:08.991054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:08.991054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:08.991054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:08.991054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:08.992087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:08.992087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:08.992087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:08.992087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:08.992087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:08.992087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:08.992087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:08.992087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:08.992087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:08.992087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:08.992087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:08.992087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:08.992087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:08.992087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:08.992087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:08.992087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:08.992087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:08.992087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:08.993047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:08.993047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:08.993047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:08.993047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:08.993047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:08.993047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:08.993047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:08.993047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:08.993047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 16:32:08.993047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:08.993047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 282/580] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:32:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:08.994370 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a620482-9459-4c5e-9e91-b6a467009e59 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:08.996762 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb70d1-b822-4b08-9082-3632c30bc82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:08.997648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb70d1-b822-4b08-9082-3632c30bc82c None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:08.998074 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb70d1-b822-4b08-9082-3632c30bc82c None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:08.998486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb70d1-b822-4b08-9082-3632c30bc82c None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:08.999695 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb70d1-b822-4b08-9082-3632c30bc82c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:09.000166 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb70d1-b822-4b08-9082-3632c30bc82c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:09.001871 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb70d1-b822-4b08-9082-3632c30bc82c None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:09.023582 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 299/581] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:09.024575 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb70d1-b822-4b08-9082-3632c30bc82c None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:09.044848 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:09.173123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a620482-9459-4c5e-9e91-b6a467009e59 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:09.186019 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb70d1-b822-4b08-9082-3632c30bc82c None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:09.236547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f601536-17ee-431b-8f0a-68785aadd52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:09.237113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f601536-17ee-431b-8f0a-68785aadd52b None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:09.237439 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f601536-17ee-431b-8f0a-68785aadd52b None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:09.237826 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f601536-17ee-431b-8f0a-68785aadd52b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:09.247505 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f601536-17ee-431b-8f0a-68785aadd52b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:09.249059 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f601536-17ee-431b-8f0a-68785aadd52b None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:09.260343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb8b44-883d-4ea9-ae1d-e0480fdea1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:09.261167 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb8b44-883d-4ea9-ae1d-e0480fdea1f7 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:09.261621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb8b44-883d-4ea9-ae1d-e0480fdea1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:09.261815 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb8b44-883d-4ea9-ae1d-e0480fdea1f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:09.262396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb8b44-883d-4ea9-ae1d-e0480fdea1f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:09.262847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb8b44-883d-4ea9-ae1d-e0480fdea1f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:09.264290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb8b44-883d-4ea9-ae1d-e0480fdea1f7 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:09.273312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-2f601536-17ee-431b-8f0a-68785aadd52b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:09.273312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:09.273312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:09.273312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:09.273312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:09.273312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:09.273312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:09.273312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:09.273312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:09.273312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:09.273312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:09.273312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:09.273312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:09.273312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:09.273312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:09.275694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:09.275694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:09.275694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:09.275694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:09.275694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:09.275694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:09.275694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:09.275694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:09.275694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:09.275694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:09.275694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:09.275694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:09.275694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:09.275694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:09.275694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:09.275694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:09.275694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:09.277415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:09.277415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:09.277415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:09.277415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:09.277415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:09.277415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:09.277415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:09.277415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:09.280097 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 300/582] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Aug 8 16:32:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:09.282866 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb8b44-883d-4ea9-ae1d-e0480fdea1f7 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:09.283077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 283/583] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:09.283361 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f601536-17ee-431b-8f0a-68785aadd52b None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:09.303814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:09.429779 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:09.449756 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f601536-17ee-431b-8f0a-68785aadd52b None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:09.474801 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb8b44-883d-4ea9-ae1d-e0480fdea1f7 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:09.491797 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793b003-513e-4e28-aa7a-f79e209e6a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:09.492346 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793b003-513e-4e28-aa7a-f79e209e6a9b None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:09.492631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793b003-513e-4e28-aa7a-f79e209e6a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:09.492920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793b003-513e-4e28-aa7a-f79e209e6a9b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:09.503876 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a793b003-513e-4e28-aa7a-f79e209e6a9b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:09.505637 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a793b003-513e-4e28-aa7a-f79e209e6a9b None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:09.520186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-a793b003-513e-4e28-aa7a-f79e209e6a9b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 16:32:09.520186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:09.520186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:09.520186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:09.520186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:09.520186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:09.520186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:09.520186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:09.520186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:09.520186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:09.520186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:09.520186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:09.520186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:09.520186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:09.520186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:09.520186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:09.522089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:09.522089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:09.522089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:09.522089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:09.522089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:09.522089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:09.522089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:09.522089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:09.522089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:09.522089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:09.522089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:09.522089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:09.522089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:09.522089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:09.522089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:09.522089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:09.522089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:09.522089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:09.525243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:09.525243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:09.525243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:09.525243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:09.525243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:09.525243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:09.525243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:09.525243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:09.525243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 16:32:09.525243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:09.527666 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 284/584] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:32:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:09.528689 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793b003-513e-4e28-aa7a-f79e209e6a9b None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:09.562041 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb4918-a68e-435d-9ab6-434f0e20b47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:09.564012 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb4918-a68e-435d-9ab6-434f0e20b47f None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:09.564627 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb4918-a68e-435d-9ab6-434f0e20b47f None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:09.565295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb4918-a68e-435d-9ab6-434f0e20b47f None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:09.567142 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb4918-a68e-435d-9ab6-434f0e20b47f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:09.567968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb4918-a68e-435d-9ab6-434f0e20b47f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:09.570874 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb4918-a68e-435d-9ab6-434f0e20b47f None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:09.602140 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 301/585] 10.222.0.13 () {56 vars in 1139 bytes} [Mon Aug 8 16:32:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:09.604449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb4918-a68e-435d-9ab6-434f0e20b47f None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:09.699782 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793b003-513e-4e28-aa7a-f79e209e6a9b None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:09.740159 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3780682-4b9a-4aa5-982f-4370d1636fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:09.740796 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3780682-4b9a-4aa5-982f-4370d1636fbd None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:09.741067 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3780682-4b9a-4aa5-982f-4370d1636fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:09.741331 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3780682-4b9a-4aa5-982f-4370d1636fbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:09.746086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:09.754663 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3780682-4b9a-4aa5-982f-4370d1636fbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:09.756290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3780682-4b9a-4aa5-982f-4370d1636fbd None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:09.767883 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:09.781287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-a3780682-4b9a-4aa5-982f-4370d1636fbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:09.781287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:09.781287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:09.781287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:09.781287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:09.781287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:09.781287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:09.781287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:09.781287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:09.781287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:09.781287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:09.781287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:09.781287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:09.781287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:09.781287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:09.781287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:09.783074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:09.783074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:09.783074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:09.783074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:09.783074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:09.783074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:09.783074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:09.783074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:09.783074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:09.783074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:09.783074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:09.783074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:09.783074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:09.783074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:09.783074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:09.783074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:09.783074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:09.794206 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:09.794206 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:09.794206 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:09.794206 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 285/586] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:09.794206 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3780682-4b9a-4aa5-982f-4370d1636fbd None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:09.824795 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb4918-a68e-435d-9ab6-434f0e20b47f None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:09.895004 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd0579-089b-4c0f-8f0c-a97c408da332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:09.902602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd0579-089b-4c0f-8f0c-a97c408da332 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:09.902602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd0579-089b-4c0f-8f0c-a97c408da332 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:09.902602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd0579-089b-4c0f-8f0c-a97c408da332 None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:09.902602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd0579-089b-4c0f-8f0c-a97c408da332 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:09.902602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd0579-089b-4c0f-8f0c-a97c408da332 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:09.904379 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd0579-089b-4c0f-8f0c-a97c408da332 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:09.935588 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 302/587] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:09.936388 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3780682-4b9a-4aa5-982f-4370d1636fbd None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:09.947290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd0579-089b-4c0f-8f0c-a97c408da332 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:09.990169 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f59370-3309-4a6c-a668-976c732c0a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:09.991006 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f59370-3309-4a6c-a668-976c732c0a99 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:09.991385 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f59370-3309-4a6c-a668-976c732c0a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:09.991770 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f59370-3309-4a6c-a668-976c732c0a99 None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:09.992780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f59370-3309-4a6c-a668-976c732c0a99 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:09.993183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f59370-3309-4a6c-a668-976c732c0a99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:10.001331 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f59370-3309-4a6c-a668-976c732c0a99 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:10.036640 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 286/588] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Aug 8 16:32:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:10.038332 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f59370-3309-4a6c-a668-976c732c0a99 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:10.077361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd0579-089b-4c0f-8f0c-a97c408da332 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:10.126329 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:10.129932 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:10.135402 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94639ed2-5198-4576-8ec5-18d9542ba93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:10.152110 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94639ed2-5198-4576-8ec5-18d9542ba93a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:10.152110 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94639ed2-5198-4576-8ec5-18d9542ba93a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:10.152110 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94639ed2-5198-4576-8ec5-18d9542ba93a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:10.152110 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94639ed2-5198-4576-8ec5-18d9542ba93a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:10.152110 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94639ed2-5198-4576-8ec5-18d9542ba93a None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:10.168475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-94639ed2-5198-4576-8ec5-18d9542ba93a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:10.168475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:10.168475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:10.168475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:10.168475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:10.168475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:10.168475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:10.168475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:10.168475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:10.168475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:10.168475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:10.168475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:10.168475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:10.168475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:10.168475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:10.168475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:10.170682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:10.170682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:10.170682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:10.170682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:10.170682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:10.170682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:10.170682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:10.170682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:10.170682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:10.170682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:10.170682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:10.170682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:10.170682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:10.170682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:10.170682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:10.170682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:10.170682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:10.173210 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:10.173210 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:10.173210 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:10.173210 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 303/589] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:10.173210 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94639ed2-5198-4576-8ec5-18d9542ba93a None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:10.213170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f59370-3309-4a6c-a668-976c732c0a99 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:10.270265 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c4730-c73b-4681-9599-9843279844ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:10.270824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c4730-c73b-4681-9599-9843279844ca None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:10.271158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c4730-c73b-4681-9599-9843279844ca None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:10.271423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c4730-c73b-4681-9599-9843279844ca None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:10.272118 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c4730-c73b-4681-9599-9843279844ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:10.272469 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c4730-c73b-4681-9599-9843279844ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:10.274993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c4730-c73b-4681-9599-9843279844ca None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:10.292457 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 287/590] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:10.293374 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c4730-c73b-4681-9599-9843279844ca None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:10.333709 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94639ed2-5198-4576-8ec5-18d9542ba93a None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:10.403117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bfd82c-b44d-4d1e-abf8-57fdc82449f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:10.404970 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bfd82c-b44d-4d1e-abf8-57fdc82449f4 None admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:10.405465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bfd82c-b44d-4d1e-abf8-57fdc82449f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:10.406364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bfd82c-b44d-4d1e-abf8-57fdc82449f4 None admin] PATH_INFO: `/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/9daa7489bced467984d39e8b8ac49d13/roles/7764573472d34ecc8bedf7a9c7935170` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:10.423329 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c4730-c73b-4681-9599-9843279844ca None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:10.442316 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bfd82c-b44d-4d1e-abf8-57fdc82449f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=56ab2fc79d964f4d86edefb4d1b2c993, user_id=9daa7489bced467984d39e8b8ac49d13, role_id=7764573472d34ecc8bedf7a9c7935170)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:10.444738 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:10.446858 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bfd82c-b44d-4d1e-abf8-57fdc82449f4 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:10.468626 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca21171-64cc-477c-83d7-f05397d717a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:10.469674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca21171-64cc-477c-83d7-f05397d717a2 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:10.470196 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca21171-64cc-477c-83d7-f05397d717a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:10.470674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca21171-64cc-477c-83d7-f05397d717a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:10.485245 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca21171-64cc-477c-83d7-f05397d717a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:10.486843 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca21171-64cc-477c-83d7-f05397d717a2 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:10.513449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-bca21171-64cc-477c-83d7-f05397d717a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:10.513449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:10.513449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:10.513449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:10.513449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:10.513449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:10.513449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:10.513449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:10.513449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:10.513449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:10.513449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:10.513449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:10.513449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:10.513449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:10.513449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:10.513449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:10.514725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:10.514725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:10.514725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:10.514725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:10.514725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:10.514725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:10.514725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:10.514725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:10.514725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:10.514725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:10.514725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:10.514725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:10.514725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:10.514725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:10.514725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:10.514725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:10.514725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:10.515958 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:10.515958 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:10.515958 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:10.515958 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 288/591] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:10.517267 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca21171-64cc-477c-83d7-f05397d717a2 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:10.574686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 304/592] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:32:10 2022] PUT /identity/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/9daa7489bced467984d39e8b8ac49d13/roles/7764573472d34ecc8bedf7a9c7935170 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:32:10.574686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bfd82c-b44d-4d1e-abf8-57fdc82449f4 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:10.636767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:10.750138 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca21171-64cc-477c-83d7-f05397d717a2 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:10.801465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bfd82c-b44d-4d1e-abf8-57fdc82449f4 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:10.833259 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ea832-ac90-4416-9782-68fc80c8e6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:10.834819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ea832-ac90-4416-9782-68fc80c8e6a6 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:10.835608 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ea832-ac90-4416-9782-68fc80c8e6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:10.836676 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ea832-ac90-4416-9782-68fc80c8e6a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:10.838233 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ea832-ac90-4416-9782-68fc80c8e6a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:10.839062 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ea832-ac90-4416-9782-68fc80c8e6a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:10.842708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ea832-ac90-4416-9782-68fc80c8e6a6 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:10.872567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 289/593] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:10.874596 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ea832-ac90-4416-9782-68fc80c8e6a6 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:10.888148 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b29f1-060f-4caa-86e7-ddead9ac5298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:10.889157 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b29f1-060f-4caa-86e7-ddead9ac5298 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:10.889547 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b29f1-060f-4caa-86e7-ddead9ac5298 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:10.889909 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b29f1-060f-4caa-86e7-ddead9ac5298 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:10.914816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b29f1-060f-4caa-86e7-ddead9ac5298 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:10.923145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b29f1-060f-4caa-86e7-ddead9ac5298 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:10.944645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-c91b29f1-060f-4caa-86e7-ddead9ac5298 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 16:32:10.944645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:10.944645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:10.944645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:10.944645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:10.944645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:10.944645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:10.944645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:10.944645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:10.944645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:10.944645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:10.944645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:10.944645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:10.944645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:10.944645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:10.944645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:10.947926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:10.947926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:10.947926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:10.947926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:10.947926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:10.947926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:10.947926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:10.947926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:10.947926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:10.947926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:10.947926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:10.947926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:10.947926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:10.947926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:10.947926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:10.947926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:10.947926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:10.947926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:10.950804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:10.950804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:10.950804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:10.950804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:10.950804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:10.950804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:10.950804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:10.950804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:10.950804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 16:32:10.950804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:10.950804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 305/594] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:32:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:10.950804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b29f1-060f-4caa-86e7-ddead9ac5298 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:10.971536 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:11.096856 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:11.107263 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b29f1-060f-4caa-86e7-ddead9ac5298 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:11.124974 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ea832-ac90-4416-9782-68fc80c8e6a6 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:11.156773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64b814-4814-48b1-b936-c6550aeb5b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:11.157782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64b814-4814-48b1-b936-c6550aeb5b2a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:11.158258 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64b814-4814-48b1-b936-c6550aeb5b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:11.158774 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64b814-4814-48b1-b936-c6550aeb5b2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:11.168606 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64b814-4814-48b1-b936-c6550aeb5b2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:11.170151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64b814-4814-48b1-b936-c6550aeb5b2a None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:11.183418 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-1d64b814-4814-48b1-b936-c6550aeb5b2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:11.183418 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:11.183418 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:11.183418 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:11.183418 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:11.183418 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:11.183418 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:11.183418 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:11.183418 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:11.183418 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:11.183418 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:11.183418 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:11.183418 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:11.183418 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:11.183418 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:11.184381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:11.184381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:11.184381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:11.184381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:11.184381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:11.184381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:11.184381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:11.184381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:11.184381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:11.184381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:11.184381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:11.184381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:11.184381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:11.184381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:11.184381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:11.184381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:11.184381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:11.185313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:11.185313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:11.185313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:11.185313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:11.185313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:11.185313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:11.185313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:11.185313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:11.187694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 306/595] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:11.188522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64b814-4814-48b1-b936-c6550aeb5b2a None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:11.194133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a12b2d-3cd3-4798-8ad7-23b790f5d3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:11.195152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a12b2d-3cd3-4798-8ad7-23b790f5d3e7 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:11.195637 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a12b2d-3cd3-4798-8ad7-23b790f5d3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:11.196102 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a12b2d-3cd3-4798-8ad7-23b790f5d3e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:11.197401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a12b2d-3cd3-4798-8ad7-23b790f5d3e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:11.198062 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a12b2d-3cd3-4798-8ad7-23b790f5d3e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:11.200899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a12b2d-3cd3-4798-8ad7-23b790f5d3e7 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:11.230800 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 290/596] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:11.232281 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c8b88-f889-4331-bfeb-6c87de6dc18f None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:11.233152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c8b88-f889-4331-bfeb-6c87de6dc18f None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:11.233455 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c8b88-f889-4331-bfeb-6c87de6dc18f None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:11.239532 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 291/597] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:11.240515 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c8b88-f889-4331-bfeb-6c87de6dc18f None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:11.335166 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64b814-4814-48b1-b936-c6550aeb5b2a None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:11.385226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422782d-28be-4fe2-bc2b-290b324f7b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:11.386194 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422782d-28be-4fe2-bc2b-290b324f7b78 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:11.386644 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422782d-28be-4fe2-bc2b-290b324f7b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:11.387076 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422782d-28be-4fe2-bc2b-290b324f7b78 None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:11.388143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422782d-28be-4fe2-bc2b-290b324f7b78 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:11.388683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422782d-28be-4fe2-bc2b-290b324f7b78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:11.390257 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422782d-28be-4fe2-bc2b-290b324f7b78 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:11.402666 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:11.415840 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 307/598] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Aug 8 16:32:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:11.417592 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422782d-28be-4fe2-bc2b-290b324f7b78 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:11.426774 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c8b88-f889-4331-bfeb-6c87de6dc18f None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:11.449817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:11.484064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb079d8b-08e8-4684-9d69-fa6b69611bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:11.485593 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb079d8b-08e8-4684-9d69-fa6b69611bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:11.486377 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb079d8b-08e8-4684-9d69-fa6b69611bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:11.487303 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb079d8b-08e8-4684-9d69-fa6b69611bf9 None admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:11.488904 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb079d8b-08e8-4684-9d69-fa6b69611bf9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:11.489715 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb079d8b-08e8-4684-9d69-fa6b69611bf9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:11.492802 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb079d8b-08e8-4684-9d69-fa6b69611bf9 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:11.542454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 292/599] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:11.545274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f710b4b-9be4-4af9-9238-b5c7943389d1 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:11.546045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f710b4b-9be4-4af9-9238-b5c7943389d1 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:11.547069 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f710b4b-9be4-4af9-9238-b5c7943389d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:11.569597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422782d-28be-4fe2-bc2b-290b324f7b78 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:11.586795 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-7f710b4b-9be4-4af9-9238-b5c7943389d1 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:11.590627 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-7f710b4b-9be4-4af9-9238-b5c7943389d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:11.631098 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a34b90-d396-4896-bf36-f4932630c6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:11.631822 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a34b90-d396-4896-bf36-f4932630c6b3 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:11.632100 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a34b90-d396-4896-bf36-f4932630c6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:11.632504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a34b90-d396-4896-bf36-f4932630c6b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:11.644523 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a34b90-d396-4896-bf36-f4932630c6b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:11.646016 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a34b90-d396-4896-bf36-f4932630c6b3 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:11.662461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-12a34b90-d396-4896-bf36-f4932630c6b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:11.662461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:11.662461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:11.662461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:11.662461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:11.662461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:11.662461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:11.662461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:11.662461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:11.662461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:11.662461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:11.662461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:11.662461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:11.662461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:11.662461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:11.663552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:11.663552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:11.663552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:11.663552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:11.663552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:11.663552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:11.663552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:11.663552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:11.663552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:11.663552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:11.663552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:11.663552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:11.663552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:11.663552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:11.663552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:11.663552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:11.663552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:11.664519 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:11.664519 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:11.664519 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:11.664519 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:11.664519 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:11.664519 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:11.664519 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:11.664519 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:11.664519 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 308/600] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:11.664519 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a34b90-d396-4896-bf36-f4932630c6b3 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:11.734629 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 293/601] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:32:11.734629 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f710b4b-9be4-4af9-9238-b5c7943389d1 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:11.776527 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:11.811234 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a34b90-d396-4896-bf36-f4932630c6b3 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:11.873170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:11.874799 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279dcf7-c3c5-4069-875a-d55825fc4df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:11.875498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279dcf7-c3c5-4069-875a-d55825fc4df2 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:11.875770 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279dcf7-c3c5-4069-875a-d55825fc4df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:11.876202 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279dcf7-c3c5-4069-875a-d55825fc4df2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:11.896824 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279dcf7-c3c5-4069-875a-d55825fc4df2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:11.901612 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279dcf7-c3c5-4069-875a-d55825fc4df2 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:11.903539 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f710b4b-9be4-4af9-9238-b5c7943389d1 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:11.940428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-4279dcf7-c3c5-4069-875a-d55825fc4df2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:11.940428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:11.940428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:11.940428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:11.940428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:11.940428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:11.940428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:11.940428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:11.940428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:11.940428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:11.940428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:11.940428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:11.940428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:11.940428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:11.940428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:11.940428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:11.942367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:11.942367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:11.942367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:11.942367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:11.942367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:11.942367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:11.942367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:11.942367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:11.942367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:11.942367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:11.942367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:11.942367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:11.942367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:11.942367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:11.942367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:11.942367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:11.942367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:11.945893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:11.945893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:11.945893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:11.945893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 309/602] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:11.945893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279dcf7-c3c5-4069-875a-d55825fc4df2 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:11.978423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a3493-69b6-4229-8131-009bd07ebfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:11.979395 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a3493-69b6-4229-8131-009bd07ebfee None admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:11.979395 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a3493-69b6-4229-8131-009bd07ebfee None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:11.979708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a3493-69b6-4229-8131-009bd07ebfee None admin] PATH_INFO: `/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/9482bf89c1b648c1a91ee00ee5fbba63/roles/7764573472d34ecc8bedf7a9c7935170` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:12.004085 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a3493-69b6-4229-8131-009bd07ebfee None admin] RBAC: Authorizing `identity:create_grant(project_id=56ab2fc79d964f4d86edefb4d1b2c993, user_id=9482bf89c1b648c1a91ee00ee5fbba63, role_id=7764573472d34ecc8bedf7a9c7935170)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:12.007400 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a3493-69b6-4229-8131-009bd07ebfee None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:12.079804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279dcf7-c3c5-4069-875a-d55825fc4df2 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:12.107425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 294/603] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:32:11 2022] PUT /identity/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/9482bf89c1b648c1a91ee00ee5fbba63/roles/7764573472d34ecc8bedf7a9c7935170 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:32:12.108743 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a3493-69b6-4229-8131-009bd07ebfee None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:12.145425 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:12.174636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128b1ca-55f5-4232-b6c0-8a5b566ace16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:12.178599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128b1ca-55f5-4232-b6c0-8a5b566ace16 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:12.178599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128b1ca-55f5-4232-b6c0-8a5b566ace16 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:12.178599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128b1ca-55f5-4232-b6c0-8a5b566ace16 None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:12.178599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128b1ca-55f5-4232-b6c0-8a5b566ace16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:12.178599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128b1ca-55f5-4232-b6c0-8a5b566ace16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:12.183680 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128b1ca-55f5-4232-b6c0-8a5b566ace16 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:12.219996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 310/604] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:12.221818 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128b1ca-55f5-4232-b6c0-8a5b566ace16 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:12.325533 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a3493-69b6-4229-8131-009bd07ebfee None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:12.367218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:12.380557 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb577fc3-9b6d-41a0-a715-2b7cafc52171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:12.383072 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb577fc3-9b6d-41a0-a715-2b7cafc52171 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:12.383072 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb577fc3-9b6d-41a0-a715-2b7cafc52171 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:12.383072 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb577fc3-9b6d-41a0-a715-2b7cafc52171 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:12.402210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb577fc3-9b6d-41a0-a715-2b7cafc52171 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:32:12.408673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-fb577fc3-9b6d-41a0-a715-2b7cafc52171 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:12.408673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:12.408673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:12.408673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:12.408673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:12.408673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:12.408673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:12.408673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:12.408673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:12.408673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:12.408673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:12.408673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:12.408673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:32:12.408673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:32:12.408673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:32:12.408673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:32:12.409708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:32:12.409708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:32:12.409708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:32:12.409708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:32:12.409708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:12.409708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:12.409708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:12.409708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:12.409708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:12.409708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:12.409708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:12.409708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:12.409708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:12.409708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:12.409708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:12.409708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:12.410586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:12.410586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:12.410586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:12.410586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:12.410586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:32:12.410586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:32:12.410586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:32:12.410586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:32:12.410586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:32:12.410586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:32:12.410586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:12.410586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:12.410586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 295/605] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:32:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:12.411369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb577fc3-9b6d-41a0-a715-2b7cafc52171 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:12.502308 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128b1ca-55f5-4232-b6c0-8a5b566ace16 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:12.549616 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb577fc3-9b6d-41a0-a715-2b7cafc52171 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:12.595287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f4f03-ea1a-4d03-98b8-621dfa5fc8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:12.596129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f4f03-ea1a-4d03-98b8-621dfa5fc8f7 None admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:12.596413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f4f03-ea1a-4d03-98b8-621dfa5fc8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:12.596686 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f4f03-ea1a-4d03-98b8-621dfa5fc8f7 None admin] PATH_INFO: `/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/ea51ef02128a4b63b11049eb0daef298/roles/7764573472d34ecc8bedf7a9c7935170` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:12.615021 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f4f03-ea1a-4d03-98b8-621dfa5fc8f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=56ab2fc79d964f4d86edefb4d1b2c993, user_id=ea51ef02128a4b63b11049eb0daef298, role_id=7764573472d34ecc8bedf7a9c7935170)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:12.616531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f4f03-ea1a-4d03-98b8-621dfa5fc8f7 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:12.640849 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317192c-0d33-42f5-8f69-d6b76a3c4d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:12.643187 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317192c-0d33-42f5-8f69-d6b76a3c4d2c None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:12.643187 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317192c-0d33-42f5-8f69-d6b76a3c4d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:12.643187 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317192c-0d33-42f5-8f69-d6b76a3c4d2c None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:12.644275 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317192c-0d33-42f5-8f69-d6b76a3c4d2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:12.644818 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317192c-0d33-42f5-8f69-d6b76a3c4d2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:12.647541 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317192c-0d33-42f5-8f69-d6b76a3c4d2c None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:12.665077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 311/606] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:12.665965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317192c-0d33-42f5-8f69-d6b76a3c4d2c None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:12.730595 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:12.770058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 296/607] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:32:12 2022] PUT /identity/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/ea51ef02128a4b63b11049eb0daef298/roles/7764573472d34ecc8bedf7a9c7935170 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:32:12.781093 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f4f03-ea1a-4d03-98b8-621dfa5fc8f7 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:12.822890 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317192c-0d33-42f5-8f69-d6b76a3c4d2c None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:12.874667 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7f37d-7cea-44f1-ac00-4161495436d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:12.878599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7f37d-7cea-44f1-ac00-4161495436d2 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:12.878599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7f37d-7cea-44f1-ac00-4161495436d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:12.878599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7f37d-7cea-44f1-ac00-4161495436d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:12.878599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7f37d-7cea-44f1-ac00-4161495436d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:12.878599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7f37d-7cea-44f1-ac00-4161495436d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:12.878599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7f37d-7cea-44f1-ac00-4161495436d2 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:12.890095 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 312/608] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Aug 8 16:32:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:12.895918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7f37d-7cea-44f1-ac00-4161495436d2 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:13.032075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:13.050849 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7f37d-7cea-44f1-ac00-4161495436d2 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:13.063047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:13.113731 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2528f1-087a-45bf-b183-bc8dd73815cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:13.114383 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2528f1-087a-45bf-b183-bc8dd73815cd None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:13.114699 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2528f1-087a-45bf-b183-bc8dd73815cd None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:13.114971 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2528f1-087a-45bf-b183-bc8dd73815cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:13.133318 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2528f1-087a-45bf-b183-bc8dd73815cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:13.135612 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2528f1-087a-45bf-b183-bc8dd73815cd None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:13.161364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-5b2528f1-087a-45bf-b183-bc8dd73815cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:13.161364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:13.161364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:13.161364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:13.161364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:13.161364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:13.161364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:13.161364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:13.161364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:13.161364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:13.161364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:13.161364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:13.161364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:13.161364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:13.161364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:13.161364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:13.162444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:13.162444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:13.162444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:13.162444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:13.162444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:13.162444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:13.162444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:13.162444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:13.162444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:13.162444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:13.162444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:13.162444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:13.162444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:13.162444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:13.162444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:13.162444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:13.162444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:13.163471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:13.163471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:13.163471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:13.166478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 313/609] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:13.168176 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30995a-5f06-4a5f-9c1f-23132025c2eb None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:13.168616 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30995a-5f06-4a5f-9c1f-23132025c2eb None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:13.169241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30995a-5f06-4a5f-9c1f-23132025c2eb None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:13.173339 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 314/610] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:13.174443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30995a-5f06-4a5f-9c1f-23132025c2eb None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:13.178514 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f4f03-ea1a-4d03-98b8-621dfa5fc8f7 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:13.247050 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8e6ca-4a83-4056-a3db-0e802f88526f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:13.248047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8e6ca-4a83-4056-a3db-0e802f88526f None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:13.248424 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8e6ca-4a83-4056-a3db-0e802f88526f None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:13.248789 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8e6ca-4a83-4056-a3db-0e802f88526f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:13.260254 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8e6ca-4a83-4056-a3db-0e802f88526f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:13.262609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8e6ca-4a83-4056-a3db-0e802f88526f None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:13.282493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-6ec8e6ca-4a83-4056-a3db-0e802f88526f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:13.282493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:13.282493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:13.282493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:13.282493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:13.282493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:13.282493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:13.282493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:13.282493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:13.282493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:13.282493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:13.282493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:13.282493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:13.282493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:13.282493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:13.283547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:13.283547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:13.283547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:13.283547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:13.283547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:13.283547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:13.283547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:13.283547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:13.283547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:13.283547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:13.283547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:13.283547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:13.283547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:13.283547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:13.283547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:13.283547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:13.283547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:13.284446 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:13.284446 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:13.284446 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:13.284446 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:13.284446 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:13.284446 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:13.284446 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:13.284446 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:13.287538 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 297/611] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 517 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:13.289419 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b94ad9-7634-47e3-bb30-973fab0c1401 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:13.289818 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b94ad9-7634-47e3-bb30-973fab0c1401 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:13.290847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b94ad9-7634-47e3-bb30-973fab0c1401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:13.319814 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-61b94ad9-7634-47e3-bb30-973fab0c1401 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:13.321540 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-61b94ad9-7634-47e3-bb30-973fab0c1401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:13.367438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30995a-5f06-4a5f-9c1f-23132025c2eb None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:13.441971 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c7e1a-23c0-4d9b-a2fc-7ae7f5a1229a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:13.446895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c7e1a-23c0-4d9b-a2fc-7ae7f5a1229a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:13.446895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c7e1a-23c0-4d9b-a2fc-7ae7f5a1229a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:13.446895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c7e1a-23c0-4d9b-a2fc-7ae7f5a1229a None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:13.446895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c7e1a-23c0-4d9b-a2fc-7ae7f5a1229a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:13.446895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c7e1a-23c0-4d9b-a2fc-7ae7f5a1229a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:13.446895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c7e1a-23c0-4d9b-a2fc-7ae7f5a1229a None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:13.458906 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 298/612] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:32:13.460252 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b94ad9-7634-47e3-bb30-973fab0c1401 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:13.486734 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 315/613] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:13.487825 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c7e1a-23c0-4d9b-a2fc-7ae7f5a1229a None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:13.516297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:13.526179 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:13.656732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c7e1a-23c0-4d9b-a2fc-7ae7f5a1229a None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:13.677143 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b94ad9-7634-47e3-bb30-973fab0c1401 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:13.709200 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed60de-4924-40c8-af0c-b812aa8a8c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:13.710414 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed60de-4924-40c8-af0c-b812aa8a8c26 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:13.710953 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed60de-4924-40c8-af0c-b812aa8a8c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:13.711747 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed60de-4924-40c8-af0c-b812aa8a8c26 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:13.721957 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed60de-4924-40c8-af0c-b812aa8a8c26 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:13.724604 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed60de-4924-40c8-af0c-b812aa8a8c26 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:13.742823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-eeed60de-4924-40c8-af0c-b812aa8a8c26 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:32:13.742823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:13.742823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:13.742823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:13.742823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:13.742823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:13.742823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:13.742823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:13.742823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:13.742823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:13.742823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:13.742823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:13.742823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:13.742823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:13.742823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:13.742823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:13.744458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:13.744458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:13.744458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:13.744458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:13.744458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:13.744458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:13.744458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:13.744458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:13.744458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:13.744458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:13.744458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:13.744458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:13.744458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:13.744458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:13.744458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:13.744458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:13.744458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:13.744458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:13.745449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:13.745449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:13.745449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:13.745449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:13.745449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:13.745449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:13.745449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:13.745449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:13.745449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:32:13.745449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:13.748011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 316/614] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:32:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:13.749055 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed60de-4924-40c8-af0c-b812aa8a8c26 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:13.765811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b525231-2bf7-482b-a33d-0f5f70d9267c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:13.766477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b525231-2bf7-482b-a33d-0f5f70d9267c None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:13.768300 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b525231-2bf7-482b-a33d-0f5f70d9267c None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:13.771930 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b525231-2bf7-482b-a33d-0f5f70d9267c None admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:13.772973 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b525231-2bf7-482b-a33d-0f5f70d9267c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:13.774890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b525231-2bf7-482b-a33d-0f5f70d9267c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:13.776634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b525231-2bf7-482b-a33d-0f5f70d9267c None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:13.798936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 299/615] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:13.800376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b525231-2bf7-482b-a33d-0f5f70d9267c None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:13.875879 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:13.912184 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed60de-4924-40c8-af0c-b812aa8a8c26 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:13.964304 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db474b-1521-4ade-bf7a-d3d5eb7aa18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:13.965113 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db474b-1521-4ade-bf7a-d3d5eb7aa18f None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:13.965494 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db474b-1521-4ade-bf7a-d3d5eb7aa18f None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:13.965866 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db474b-1521-4ade-bf7a-d3d5eb7aa18f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:13.986241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db474b-1521-4ade-bf7a-d3d5eb7aa18f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:32:13.992236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-37db474b-1521-4ade-bf7a-d3d5eb7aa18f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:13.992236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:13.992236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:13.992236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:13.992236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:13.992236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:13.992236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:13.992236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:13.992236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:13.992236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:13.992236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:13.992236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:13.992236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:32:13.992236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:32:13.992236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:32:13.992236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:32:13.995163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:32:13.995163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:32:13.995163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:32:13.995163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:32:13.995163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:13.995163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:13.995163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:13.995163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:13.995163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:13.995163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:13.995163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:13.995163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:13.995163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:13.995163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:13.995163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:13.995163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:13.996206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:13.996206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:13.996206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:13.996206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:13.996206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:32:13.996206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:32:13.996206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:32:13.996206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:32:13.996206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:32:13.996206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:32:13.996206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:13.996206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:13.998592 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 317/616] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:32:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:14.005243 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db474b-1521-4ade-bf7a-d3d5eb7aa18f None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:14.057737 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:14.077131 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b525231-2bf7-482b-a33d-0f5f70d9267c None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:14.133135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c0a4a-8bc3-46ae-8292-9cc0aa662e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:14.135338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c0a4a-8bc3-46ae-8292-9cc0aa662e55 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:14.136800 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c0a4a-8bc3-46ae-8292-9cc0aa662e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:14.137408 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c0a4a-8bc3-46ae-8292-9cc0aa662e55 None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:14.138391 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c0a4a-8bc3-46ae-8292-9cc0aa662e55 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:14.139955 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c0a4a-8bc3-46ae-8292-9cc0aa662e55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:14.142410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c0a4a-8bc3-46ae-8292-9cc0aa662e55 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:14.178759 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 300/617] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Aug 8 16:32:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:14.179937 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c0a4a-8bc3-46ae-8292-9cc0aa662e55 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:14.246258 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db474b-1521-4ade-bf7a-d3d5eb7aa18f None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:14.349410 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77eb36a-51c1-4d3d-b12f-564460c46a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:14.350537 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77eb36a-51c1-4d3d-b12f-564460c46a87 None admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:14.351356 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77eb36a-51c1-4d3d-b12f-564460c46a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:14.351839 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77eb36a-51c1-4d3d-b12f-564460c46a87 None admin] PATH_INFO: `/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/7bbf0678f34e46e8b56aaeabce955180/roles/7764573472d34ecc8bedf7a9c7935170` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:14.366302 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c0a4a-8bc3-46ae-8292-9cc0aa662e55 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:14.385834 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:14.408186 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77eb36a-51c1-4d3d-b12f-564460c46a87 None admin] RBAC: Authorizing `identity:create_grant(project_id=56ab2fc79d964f4d86edefb4d1b2c993, user_id=7bbf0678f34e46e8b56aaeabce955180, role_id=7764573472d34ecc8bedf7a9c7935170)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:14.411410 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77eb36a-51c1-4d3d-b12f-564460c46a87 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:14.442501 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194cb78f-fdcf-4354-893f-76f42a437b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:14.442501 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cb78f-fdcf-4354-893f-76f42a437b88 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:14.442501 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cb78f-fdcf-4354-893f-76f42a437b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:14.442501 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cb78f-fdcf-4354-893f-76f42a437b88 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:14.443077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194cb78f-fdcf-4354-893f-76f42a437b88 None admin] RBAC: Adding query filter params (name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:14.443077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194cb78f-fdcf-4354-893f-76f42a437b88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:14.443077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194cb78f-fdcf-4354-893f-76f42a437b88 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:14.453579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 301/618] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Aug 8 16:32:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:14.453579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194cb78f-fdcf-4354-893f-76f42a437b88 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:14.465700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:14.511179 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 318/619] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:32:13 2022] PUT /identity/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/7bbf0678f34e46e8b56aaeabce955180/roles/7764573472d34ecc8bedf7a9c7935170 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:32:14.512059 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77eb36a-51c1-4d3d-b12f-564460c46a87 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:14.600589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194cb78f-fdcf-4354-893f-76f42a437b88 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:14.672035 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e1a32-cdd3-4b4f-b556-1023f39ff0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:14.673173 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e1a32-cdd3-4b4f-b556-1023f39ff0d7 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:14.673831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e1a32-cdd3-4b4f-b556-1023f39ff0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:14.674077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77eb36a-51c1-4d3d-b12f-564460c46a87 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:14.674716 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e1a32-cdd3-4b4f-b556-1023f39ff0d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:14.691822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e1a32-cdd3-4b4f-b556-1023f39ff0d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:14.694070 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e1a32-cdd3-4b4f-b556-1023f39ff0d7 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:14.717073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-d43e1a32-cdd3-4b4f-b556-1023f39ff0d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:14.717073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:14.717073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:14.717073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:14.717073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:14.717073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:14.717073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:14.717073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:14.717073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:14.717073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:14.717073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:14.717073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:14.717073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:14.717073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:14.717073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:14.717073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:14.718411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:14.718411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:14.718411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:14.718411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:14.718411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:14.718411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:14.718411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:14.718411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:14.718411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:14.718411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:14.718411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:14.718411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:14.718411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:14.718411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:14.718411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:14.718411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:14.718411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:14.719467 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:14.719467 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:14.719467 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:14.726712 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 302/620] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:14.729091 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3487c1-c0e7-4018-9b81-6693ae68e7a7 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:14.729820 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3487c1-c0e7-4018-9b81-6693ae68e7a7 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:14.730360 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3487c1-c0e7-4018-9b81-6693ae68e7a7 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:14.737230 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 303/621] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:14.738530 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3487c1-c0e7-4018-9b81-6693ae68e7a7 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:14.744105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10629547-ac04-4665-8008-30007d0fc6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:14.744774 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10629547-ac04-4665-8008-30007d0fc6e7 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:14.745222 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10629547-ac04-4665-8008-30007d0fc6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:14.745501 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10629547-ac04-4665-8008-30007d0fc6e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:14.761567 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10629547-ac04-4665-8008-30007d0fc6e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:14.763788 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10629547-ac04-4665-8008-30007d0fc6e7 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:14.786232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-10629547-ac04-4665-8008-30007d0fc6e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:14.786232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:14.786232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:14.786232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:14.786232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:14.786232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:14.786232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:14.786232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:14.786232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:14.786232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:14.786232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:14.786232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:14.786232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:14.786232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:14.786232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:14.786232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:14.787393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:14.787393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:14.787393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:14.787393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:14.787393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:14.787393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:14.787393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:14.787393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:14.787393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:14.787393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:14.787393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:14.787393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:14.787393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:14.787393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:14.787393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:14.787393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:14.787393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:14.788295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:14.788295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:14.788295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:14.788956 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 319/622] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:14.791173 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d11e27-57b9-4dd4-a5f7-246209c89fc0 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:14.791525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d11e27-57b9-4dd4-a5f7-246209c89fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:14.791766 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d11e27-57b9-4dd4-a5f7-246209c89fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:14.818204 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-72d11e27-57b9-4dd4-a5f7-246209c89fc0 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:14.819614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-72d11e27-57b9-4dd4-a5f7-246209c89fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:14.825768 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:14.887029 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:14.903398 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3487c1-c0e7-4018-9b81-6693ae68e7a7 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:14.928897 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 320/623] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:32:14.935797 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d11e27-57b9-4dd4-a5f7-246209c89fc0 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:14.967739 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad232e1-49da-4755-b7b9-3e4e33a8111c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:14.971771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad232e1-49da-4755-b7b9-3e4e33a8111c None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:14.972087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad232e1-49da-4755-b7b9-3e4e33a8111c None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:14.972815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad232e1-49da-4755-b7b9-3e4e33a8111c None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:14.973641 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad232e1-49da-4755-b7b9-3e4e33a8111c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:14.975345 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad232e1-49da-4755-b7b9-3e4e33a8111c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:14.978925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad232e1-49da-4755-b7b9-3e4e33a8111c None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:15.004136 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 304/624] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:15.009964 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad232e1-49da-4755-b7b9-3e4e33a8111c None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:15.097576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d11e27-57b9-4dd4-a5f7-246209c89fc0 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:15.158365 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fba41-d4de-42e7-8845-67e31c7ab750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:15.160956 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fba41-d4de-42e7-8845-67e31c7ab750 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:15.163551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fba41-d4de-42e7-8845-67e31c7ab750 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:15.164088 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fba41-d4de-42e7-8845-67e31c7ab750 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:15.165153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fba41-d4de-42e7-8845-67e31c7ab750 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:15.167307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fba41-d4de-42e7-8845-67e31c7ab750 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:15.168985 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fba41-d4de-42e7-8845-67e31c7ab750 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:15.187531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad232e1-49da-4755-b7b9-3e4e33a8111c None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:15.204477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 321/625] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:15.205730 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fba41-d4de-42e7-8845-67e31c7ab750 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:15.236506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:15.255932 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:15.276807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8cf6c-ad34-4c38-a522-67d60d4440ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:15.278619 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8cf6c-ad34-4c38-a522-67d60d4440ce None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:15.278878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8cf6c-ad34-4c38-a522-67d60d4440ce None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:15.279153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8cf6c-ad34-4c38-a522-67d60d4440ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:15.296475 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c8cf6c-ad34-4c38-a522-67d60d4440ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:32:15.303109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-76c8cf6c-ad34-4c38-a522-67d60d4440ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:15.303109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:15.303109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:15.303109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:15.303109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:15.303109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:15.303109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:15.303109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:15.303109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:15.303109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:15.303109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:15.303109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:15.303109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:32:15.303109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:32:15.303109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:32:15.303109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:32:15.305291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:32:15.305291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:32:15.305291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:32:15.305291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:32:15.305291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:15.305291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:15.305291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:15.305291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:15.305291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:15.305291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:15.305291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:15.305291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:15.305291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:15.305291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:15.305291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:15.305291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:15.307678 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:15.307678 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:15.307678 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:15.307678 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:15.307678 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:32:15.307678 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:32:15.307678 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:32:15.307678 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:32:15.307678 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:32:15.307678 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:32:15.307678 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:15.307678 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:15.310003 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 305/626] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:32:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:15.312284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8cf6c-ad34-4c38-a522-67d60d4440ce None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:15.375107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fba41-d4de-42e7-8845-67e31c7ab750 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:15.433103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5304e0f-773d-447b-be9d-ac79e2fbbf2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:15.441947 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5304e0f-773d-447b-be9d-ac79e2fbbf2d None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:15.441947 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5304e0f-773d-447b-be9d-ac79e2fbbf2d None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:15.441947 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5304e0f-773d-447b-be9d-ac79e2fbbf2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:15.442726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5304e0f-773d-447b-be9d-ac79e2fbbf2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:15.444231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5304e0f-773d-447b-be9d-ac79e2fbbf2d None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:15.457853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-b5304e0f-773d-447b-be9d-ac79e2fbbf2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:15.457853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:15.457853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:15.457853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:15.457853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:15.457853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:15.457853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:15.457853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:15.457853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:15.457853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:15.457853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:15.457853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:15.457853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:15.457853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:15.457853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:15.460280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:15.460280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:15.460280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:15.460280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:15.460280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:15.460280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:15.460280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:15.460280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:15.460280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:15.460280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:15.460280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:15.460280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:15.460280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:15.460280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:15.460280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:15.460280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:15.460280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:15.462597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:15.462597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:15.462597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:15.462597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:15.462597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:15.462597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:15.462597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:15.462597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:15.462597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 322/627] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:15.462597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e194b-f37e-4aea-9dda-4880295f5822 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:15.462597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e194b-f37e-4aea-9dda-4880295f5822 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:15.463790 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e194b-f37e-4aea-9dda-4880295f5822 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:15.471228 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 323/628] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:15.471228 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e194b-f37e-4aea-9dda-4880295f5822 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:15.505293 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8cf6c-ad34-4c38-a522-67d60d4440ce None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:15.560347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e8ec35-2f1c-4e21-9e62-24d95fd17801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:15.561317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8ec35-2f1c-4e21-9e62-24d95fd17801 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:15.561962 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8ec35-2f1c-4e21-9e62-24d95fd17801 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:15.562247 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8ec35-2f1c-4e21-9e62-24d95fd17801 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:15.581439 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e8ec35-2f1c-4e21-9e62-24d95fd17801 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:15.583263 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e8ec35-2f1c-4e21-9e62-24d95fd17801 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:15.598000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-e2e8ec35-2f1c-4e21-9e62-24d95fd17801 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:32:15.598000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:15.598000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:15.598000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:15.598000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:15.598000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:15.598000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:15.598000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:15.598000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:15.598000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:15.598000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:15.598000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:15.598000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:15.598000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:15.598000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:15.598000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:15.599772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:15.599772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:15.599772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:15.599772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:15.599772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:15.599772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:15.599772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:15.599772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:15.599772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:15.599772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:15.599772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:15.599772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:15.599772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:15.599772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:15.599772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:15.599772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:15.599772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:15.599772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:15.602128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:15.602128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:15.602128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:15.602128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:15.602128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:15.602128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:15.602128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:15.602128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:15.602128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:32:15.602128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:15.602128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 306/629] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Aug 8 16:32:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:15.602128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e8ec35-2f1c-4e21-9e62-24d95fd17801 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:15.616256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:15.633440 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e194b-f37e-4aea-9dda-4880295f5822 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:15.673647 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:15.680419 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e7869f-b20b-488d-ae42-3a403c0c0d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:15.681550 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7869f-b20b-488d-ae42-3a403c0c0d01 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:15.688617 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7869f-b20b-488d-ae42-3a403c0c0d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:15.688617 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7869f-b20b-488d-ae42-3a403c0c0d01 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:15.688617 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e7869f-b20b-488d-ae42-3a403c0c0d01 None admin] RBAC: Adding query filter params (name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:15.688617 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e7869f-b20b-488d-ae42-3a403c0c0d01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:15.688617 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e7869f-b20b-488d-ae42-3a403c0c0d01 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:15.700099 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 324/630] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Aug 8 16:32:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:15.701365 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787d198-f805-43d3-8737-0a273ed76133 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:15.701753 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787d198-f805-43d3-8737-0a273ed76133 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:15.702090 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787d198-f805-43d3-8737-0a273ed76133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:15.727533 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-9787d198-f805-43d3-8737-0a273ed76133 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:15.729326 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-9787d198-f805-43d3-8737-0a273ed76133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:15.803032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e8ec35-2f1c-4e21-9e62-24d95fd17801 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:15.825353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 325/631] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:32:15.825353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787d198-f805-43d3-8737-0a273ed76133 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:15.875136 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165018d-67dc-4fac-864b-b4aa12ad4d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:15.875803 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165018d-67dc-4fac-864b-b4aa12ad4d40 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:15.876181 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165018d-67dc-4fac-864b-b4aa12ad4d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:15.876358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165018d-67dc-4fac-864b-b4aa12ad4d40 None admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:15.877019 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165018d-67dc-4fac-864b-b4aa12ad4d40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:15.877337 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165018d-67dc-4fac-864b-b4aa12ad4d40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:15.878780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165018d-67dc-4fac-864b-b4aa12ad4d40 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:15.898617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 307/632] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:15.898617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165018d-67dc-4fac-864b-b4aa12ad4d40 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:15.956259 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:15.981195 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787d198-f805-43d3-8737-0a273ed76133 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:16.031257 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb9d71-b436-42c3-a7eb-900310108646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:16.031930 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb9d71-b436-42c3-a7eb-900310108646 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:16.032229 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb9d71-b436-42c3-a7eb-900310108646 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:16.032478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb9d71-b436-42c3-a7eb-900310108646 None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:16.033889 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb9d71-b436-42c3-a7eb-900310108646 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:16.034524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb9d71-b436-42c3-a7eb-900310108646 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:16.036281 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb9d71-b436-42c3-a7eb-900310108646 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:16.045089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:16.048574 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165018d-67dc-4fac-864b-b4aa12ad4d40 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:16.078086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 326/633] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Aug 8 16:32:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:16.078086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb9d71-b436-42c3-a7eb-900310108646 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:16.107125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27798e2-caf8-4f65-acc7-194b1cabd43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:16.107768 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27798e2-caf8-4f65-acc7-194b1cabd43c None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:16.108039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27798e2-caf8-4f65-acc7-194b1cabd43c None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:16.108300 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27798e2-caf8-4f65-acc7-194b1cabd43c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:16.122722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27798e2-caf8-4f65-acc7-194b1cabd43c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:16.124198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27798e2-caf8-4f65-acc7-194b1cabd43c None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:16.148250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-f27798e2-caf8-4f65-acc7-194b1cabd43c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:16.148250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:16.148250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:16.148250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:16.148250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:16.148250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:16.148250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:16.148250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:16.148250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:16.148250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:16.148250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:16.148250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:16.148250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:16.148250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:16.148250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:16.148250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:16.149411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:16.149411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:16.149411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:16.149411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:16.149411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:16.149411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:16.149411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:16.149411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:16.149411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:16.149411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:16.149411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:16.149411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:16.149411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:16.149411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:16.149411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:16.149411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:16.149411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:16.151491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:16.151491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:16.151491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:16.152076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 308/634] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:16.152974 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27798e2-caf8-4f65-acc7-194b1cabd43c None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:16.219455 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb9d71-b436-42c3-a7eb-900310108646 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:16.267712 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440977bf-cc7d-4a59-8849-17f6e411a9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:16.271477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440977bf-cc7d-4a59-8849-17f6e411a9ec None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:16.272548 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440977bf-cc7d-4a59-8849-17f6e411a9ec None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:16.272823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440977bf-cc7d-4a59-8849-17f6e411a9ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:16.291805 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440977bf-cc7d-4a59-8849-17f6e411a9ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:32:16.297390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:16.297565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27798e2-caf8-4f65-acc7-194b1cabd43c None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:16.299152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-440977bf-cc7d-4a59-8849-17f6e411a9ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:16.299152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:16.299152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:16.299152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:16.299152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:16.299152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:16.299152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:16.299152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:16.299152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:16.299152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:16.299152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:16.299152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:16.299152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:32:16.299152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:32:16.299152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:32:16.299152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:32:16.300313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:32:16.300313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:32:16.300313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:32:16.300313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:32:16.300313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:16.300313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:16.300313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:16.300313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:16.300313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:16.300313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:16.300313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:16.300313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:16.300313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:16.300313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:16.300313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:16.300313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:16.301415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:16.301415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:16.301415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:16.301415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:16.301415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:32:16.301415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:32:16.301415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:32:16.301415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:32:16.301415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:32:16.301415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:32:16.301415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:16.301415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:16.304893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 327/635] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:32:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:16.306522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440977bf-cc7d-4a59-8849-17f6e411a9ec None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:16.354762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd85526-27a2-4942-bd77-d017581284ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:16.356986 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd85526-27a2-4942-bd77-d017581284ab None admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:16.357349 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd85526-27a2-4942-bd77-d017581284ab None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:16.357734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd85526-27a2-4942-bd77-d017581284ab None admin] PATH_INFO: `/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/c606decb832444e38a194b98f8d15f8c/roles/7764573472d34ecc8bedf7a9c7935170` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:16.377857 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd85526-27a2-4942-bd77-d017581284ab None admin] RBAC: Authorizing `identity:create_grant(project_id=56ab2fc79d964f4d86edefb4d1b2c993, user_id=c606decb832444e38a194b98f8d15f8c, role_id=7764573472d34ecc8bedf7a9c7935170)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:16.379807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd85526-27a2-4942-bd77-d017581284ab None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:16.434997 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440977bf-cc7d-4a59-8849-17f6e411a9ec None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:16.458709 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 309/636] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:32:16 2022] PUT /identity/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/c606decb832444e38a194b98f8d15f8c/roles/7764573472d34ecc8bedf7a9c7935170 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:32:16.460025 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd85526-27a2-4942-bd77-d017581284ab None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:16.463919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:16.527781 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49921bd-03ff-4bd3-b6ea-bdb760905b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:16.528430 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49921bd-03ff-4bd3-b6ea-bdb760905b76 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:16.528709 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49921bd-03ff-4bd3-b6ea-bdb760905b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:16.528982 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49921bd-03ff-4bd3-b6ea-bdb760905b76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:16.652654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd85526-27a2-4942-bd77-d017581284ab None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:16.694868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:16.701233 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49921bd-03ff-4bd3-b6ea-bdb760905b76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:16.704797 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49921bd-03ff-4bd3-b6ea-bdb760905b76 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:16.716374 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07807d-496d-49ab-ace4-37feab6d2c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:16.717611 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07807d-496d-49ab-ace4-37feab6d2c95 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:16.718518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07807d-496d-49ab-ace4-37feab6d2c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:16.719405 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07807d-496d-49ab-ace4-37feab6d2c95 None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:16.720278 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07807d-496d-49ab-ace4-37feab6d2c95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:16.720855 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07807d-496d-49ab-ace4-37feab6d2c95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:16.722815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07807d-496d-49ab-ace4-37feab6d2c95 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:16.745689 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 310/637] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:16.746940 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07807d-496d-49ab-ace4-37feab6d2c95 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:16.829528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:16.885177 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 328/638] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:32:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:32:16.886224 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49921bd-03ff-4bd3-b6ea-bdb760905b76 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:16.886345 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07807d-496d-49ab-ace4-37feab6d2c95 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:16.939612 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4af21-9e4d-4695-9e2c-2ee69016ad54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:16.940279 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4af21-9e4d-4695-9e2c-2ee69016ad54 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:16.940649 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4af21-9e4d-4695-9e2c-2ee69016ad54 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:16.941021 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4af21-9e4d-4695-9e2c-2ee69016ad54 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:16.941787 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c4af21-9e4d-4695-9e2c-2ee69016ad54 None admin] RBAC: Adding query filter params (name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:16.942158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c4af21-9e4d-4695-9e2c-2ee69016ad54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:16.944941 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c4af21-9e4d-4695-9e2c-2ee69016ad54 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:16.955018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 311/639] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Aug 8 16:32:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:16.956743 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65af24-6e79-43fb-a0b9-2aa59b9c2f34 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:16.957780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65af24-6e79-43fb-a0b9-2aa59b9c2f34 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:16.958142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65af24-6e79-43fb-a0b9-2aa59b9c2f34 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:16.963785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 312/640] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:16.963785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65af24-6e79-43fb-a0b9-2aa59b9c2f34 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:17.037055 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:17.040307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49921bd-03ff-4bd3-b6ea-bdb760905b76 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:17.090589 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c2958-5629-4415-bdc3-3315626ef9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:17.091822 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c2958-5629-4415-bdc3-3315626ef9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:17.092137 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c2958-5629-4415-bdc3-3315626ef9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:17.092881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c2958-5629-4415-bdc3-3315626ef9c0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:17.103643 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c2958-5629-4415-bdc3-3315626ef9c0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:17.105099 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c2958-5629-4415-bdc3-3315626ef9c0 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:17.109048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65af24-6e79-43fb-a0b9-2aa59b9c2f34 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:17.120903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-f65c2958-5629-4415-bdc3-3315626ef9c0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 16:32:17.120903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:17.120903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:17.120903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:17.120903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:17.120903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:17.120903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:17.120903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:17.120903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:17.120903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:17.120903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:17.120903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:17.120903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:17.120903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:17.120903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:17.120903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:17.121952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:17.121952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:17.121952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:17.121952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:17.121952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:17.121952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:17.121952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:17.121952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:17.121952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:17.121952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:17.121952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:17.121952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:17.121952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:17.121952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:17.121952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:17.121952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:17.121952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:17.121952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:17.123047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:17.123047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:17.123047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:17.123047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:17.123047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:17.123047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:17.123047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:17.123047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:17.123047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 16:32:17.123047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:17.123047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 329/641] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:32:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:17.123047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c2958-5629-4415-bdc3-3315626ef9c0 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:17.144104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:17.156591 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90300b9-db63-4942-9708-983a7c2c6561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:17.157731 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90300b9-db63-4942-9708-983a7c2c6561 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:17.158136 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90300b9-db63-4942-9708-983a7c2c6561 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:17.158483 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90300b9-db63-4942-9708-983a7c2c6561 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:17.171929 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90300b9-db63-4942-9708-983a7c2c6561 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:17.173377 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90300b9-db63-4942-9708-983a7c2c6561 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:17.192032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-a90300b9-db63-4942-9708-983a7c2c6561 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:17.192032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:17.192032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:17.192032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:17.192032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:17.192032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:17.192032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:17.192032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:17.192032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:17.192032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:17.192032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:17.192032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:17.192032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:17.192032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:17.192032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:17.192032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:17.193113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:17.193113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:17.193113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:17.193113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:17.193113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:17.193113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:17.193113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:17.193113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:17.193113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:17.193113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:17.193113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:17.193113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:17.193113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:17.193113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:17.193113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:17.193113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:17.193113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:17.194047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:17.194047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:17.194047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:17.194047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 313/642] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:17.195031 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232d408-aabf-444f-abaf-2eabedd9f25f None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:17.195307 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232d408-aabf-444f-abaf-2eabedd9f25f None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:17.195576 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232d408-aabf-444f-abaf-2eabedd9f25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:17.215808 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-2232d408-aabf-444f-abaf-2eabedd9f25f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:17.216990 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-2232d408-aabf-444f-abaf-2eabedd9f25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:17.268337 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c2958-5629-4415-bdc3-3315626ef9c0 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:17.305185 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 314/643] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:32:17.306828 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232d408-aabf-444f-abaf-2eabedd9f25f None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:17.326222 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758bbfa6-3463-4fab-8084-f1ae42de328d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:17.327255 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758bbfa6-3463-4fab-8084-f1ae42de328d None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:17.327725 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758bbfa6-3463-4fab-8084-f1ae42de328d None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:17.328159 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758bbfa6-3463-4fab-8084-f1ae42de328d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:17.349048 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758bbfa6-3463-4fab-8084-f1ae42de328d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:17.351124 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758bbfa6-3463-4fab-8084-f1ae42de328d None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:17.369352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-758bbfa6-3463-4fab-8084-f1ae42de328d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:17.369352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:17.369352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:17.369352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:17.369352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:17.369352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:17.369352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:17.369352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:17.369352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:17.369352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:17.369352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:17.369352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:17.369352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:17.369352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:17.369352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:17.369352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:17.370394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:17.370394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:17.370394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:17.370394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:17.370394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:17.370394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:17.370394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:17.370394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:17.370394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:17.370394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:17.370394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:17.370394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:17.370394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:17.370394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:17.370394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:17.370394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:17.370394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:17.371576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:17.371576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:17.371576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:17.373353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 330/644] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:17.374195 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758bbfa6-3463-4fab-8084-f1ae42de328d None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:17.395804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:17.468366 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232d408-aabf-444f-abaf-2eabedd9f25f None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:17.471233 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:17.510055 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ed849-fd8d-4f47-a68a-b3d3bad9c372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:17.510928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ed849-fd8d-4f47-a68a-b3d3bad9c372 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:17.511338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ed849-fd8d-4f47-a68a-b3d3bad9c372 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:17.511725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ed849-fd8d-4f47-a68a-b3d3bad9c372 None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:17.512940 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ed849-fd8d-4f47-a68a-b3d3bad9c372 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:17.513363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ed849-fd8d-4f47-a68a-b3d3bad9c372 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:17.515345 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ed849-fd8d-4f47-a68a-b3d3bad9c372 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:17.520427 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758bbfa6-3463-4fab-8084-f1ae42de328d None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:17.558602 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 315/645] 10.222.0.13 () {56 vars in 1139 bytes} [Mon Aug 8 16:32:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:17.558602 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ed849-fd8d-4f47-a68a-b3d3bad9c372 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:17.599198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ff126-100c-4982-a29f-0aee8e8200a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:17.600222 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ff126-100c-4982-a29f-0aee8e8200a9 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:17.600679 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ff126-100c-4982-a29f-0aee8e8200a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:17.601287 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ff126-100c-4982-a29f-0aee8e8200a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:17.602373 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ff126-100c-4982-a29f-0aee8e8200a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:17.603523 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ff126-100c-4982-a29f-0aee8e8200a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:17.605964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ff126-100c-4982-a29f-0aee8e8200a9 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:17.626200 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 331/646] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:17.627158 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ff126-100c-4982-a29f-0aee8e8200a9 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:17.719808 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ed849-fd8d-4f47-a68a-b3d3bad9c372 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:17.753487 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:17.772176 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095d2b6-6abd-45e6-9462-79988aa87899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:17.773067 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095d2b6-6abd-45e6-9462-79988aa87899 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:17.773254 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095d2b6-6abd-45e6-9462-79988aa87899 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:17.773536 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095d2b6-6abd-45e6-9462-79988aa87899 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:17.789790 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3095d2b6-6abd-45e6-9462-79988aa87899 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:32:17.795448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-3095d2b6-6abd-45e6-9462-79988aa87899 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:17.795448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:17.795448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:17.795448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:17.795448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:17.795448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:17.795448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:17.795448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:17.795448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:17.795448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:17.795448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:17.795448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:17.795448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:32:17.795448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:32:17.795448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:32:17.795448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:32:17.796509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:32:17.796509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:32:17.796509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:32:17.796509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:32:17.796509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:17.796509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:17.796509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:17.796509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:17.796509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:17.796509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:17.796509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:17.796509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:17.796509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:17.796509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:17.796509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:17.796509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:17.797584 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:17.797584 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:17.797584 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:17.797584 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:17.797584 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:32:17.797584 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:32:17.797584 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:32:17.797584 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:32:17.797584 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:32:17.797584 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:32:17.797584 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:17.797584 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:17.797584 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 316/647] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:32:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:17.797584 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095d2b6-6abd-45e6-9462-79988aa87899 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:17.802741 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ff126-100c-4982-a29f-0aee8e8200a9 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:17.843144 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:17.849153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ab3a7-b0ef-45a1-adb1-aa767cdc6974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:17.849792 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ab3a7-b0ef-45a1-adb1-aa767cdc6974 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:17.850067 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ab3a7-b0ef-45a1-adb1-aa767cdc6974 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:17.850335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ab3a7-b0ef-45a1-adb1-aa767cdc6974 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:17.851048 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ab3a7-b0ef-45a1-adb1-aa767cdc6974 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:17.851346 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ab3a7-b0ef-45a1-adb1-aa767cdc6974 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:17.854440 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ab3a7-b0ef-45a1-adb1-aa767cdc6974 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:17.870492 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 332/648] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:17.871712 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ab3a7-b0ef-45a1-adb1-aa767cdc6974 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:17.969091 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095d2b6-6abd-45e6-9462-79988aa87899 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:18.017324 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ab3a7-b0ef-45a1-adb1-aa767cdc6974 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:18.022722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a467b3-1fe3-47b1-9179-484ca53ec4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:18.023351 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a467b3-1fe3-47b1-9179-484ca53ec4dd None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:18.023628 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a467b3-1fe3-47b1-9179-484ca53ec4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:18.023900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a467b3-1fe3-47b1-9179-484ca53ec4dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:18.054703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:18.061379 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53a948-52fb-49df-80e5-16c8f4e49996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:18.062111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53a948-52fb-49df-80e5-16c8f4e49996 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:18.062331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53a948-52fb-49df-80e5-16c8f4e49996 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:18.062582 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53a948-52fb-49df-80e5-16c8f4e49996 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:18.072184 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53a948-52fb-49df-80e5-16c8f4e49996 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:18.073737 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53a948-52fb-49df-80e5-16c8f4e49996 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:18.089939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-da53a948-52fb-49df-80e5-16c8f4e49996 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:18.089939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:18.089939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:18.089939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:18.089939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:18.089939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:18.089939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:18.089939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:18.089939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:18.089939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:18.089939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:18.089939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:18.089939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:18.089939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:18.089939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:18.093661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:18.093661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:18.093661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:18.093661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:18.093661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:18.093661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:18.093661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:18.093661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:18.093661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:18.093661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:18.093661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:18.093661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:18.093661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:18.093661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:18.093661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:18.093661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:18.093661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:18.094891 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:18.094891 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:18.094891 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:18.094891 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:18.094891 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:18.094891 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:18.094891 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:18.094891 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:18.096631 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 333/649] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:18.097598 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53a948-52fb-49df-80e5-16c8f4e49996 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:18.142097 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a467b3-1fe3-47b1-9179-484ca53ec4dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:18.144248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a467b3-1fe3-47b1-9179-484ca53ec4dd None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:18.179002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:18.242771 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53a948-52fb-49df-80e5-16c8f4e49996 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:18.290271 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cc92c-ede8-469e-8934-c81fbd74547a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:18.291855 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cc92c-ede8-469e-8934-c81fbd74547a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:18.292220 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cc92c-ede8-469e-8934-c81fbd74547a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:18.292820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cc92c-ede8-469e-8934-c81fbd74547a None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:18.294097 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8cc92c-ede8-469e-8934-c81fbd74547a None admin] RBAC: Adding query filter params (name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:18.294673 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8cc92c-ede8-469e-8934-c81fbd74547a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:18.297492 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8cc92c-ede8-469e-8934-c81fbd74547a None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:18.305623 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 317/650] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:32:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:32:18.305623 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a467b3-1fe3-47b1-9179-484ca53ec4dd None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:18.312452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 334/651] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Aug 8 16:32:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:18.319313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cc92c-ede8-469e-8934-c81fbd74547a None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:18.406305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:18.448507 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a467b3-1fe3-47b1-9179-484ca53ec4dd None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:18.475219 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cc92c-ede8-469e-8934-c81fbd74547a None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:18.481778 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:18.501814 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16488fa-58ac-4927-b225-f71ee1cf209f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:18.502867 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16488fa-58ac-4927-b225-f71ee1cf209f None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:18.503138 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16488fa-58ac-4927-b225-f71ee1cf209f None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:18.503495 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16488fa-58ac-4927-b225-f71ee1cf209f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:18.514756 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16488fa-58ac-4927-b225-f71ee1cf209f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:18.516499 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16488fa-58ac-4927-b225-f71ee1cf209f None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:18.520371 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf8b4b-e68b-4161-92c1-1ae543b141b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:18.521487 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf8b4b-e68b-4161-92c1-1ae543b141b6 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:18.522892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf8b4b-e68b-4161-92c1-1ae543b141b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:18.523182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf8b4b-e68b-4161-92c1-1ae543b141b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:18.523896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf8b4b-e68b-4161-92c1-1ae543b141b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:18.524202 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf8b4b-e68b-4161-92c1-1ae543b141b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:18.525761 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf8b4b-e68b-4161-92c1-1ae543b141b6 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:18.534462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-e16488fa-58ac-4927-b225-f71ee1cf209f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 16:32:18.534462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:18.534462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:18.534462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:18.534462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:18.534462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:18.534462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:18.534462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:18.534462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:18.534462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:18.534462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:18.534462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:18.534462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:18.534462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:18.534462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:18.534462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:18.535719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:18.535719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:18.535719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:18.535719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:18.535719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:18.535719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:18.535719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:18.535719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:18.535719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:18.535719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:18.535719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:18.535719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:18.535719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:18.535719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:18.535719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:18.535719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:18.535719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:18.535719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:18.536698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:18.536698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:18.536698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:18.536698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:18.536698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:18.536698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:18.536698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:18.536698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:18.536698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 16:32:18.536698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:18.542690 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 318/652] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:32:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:18.543900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16488fa-58ac-4927-b225-f71ee1cf209f None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:18.547593 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 335/653] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:18.548459 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf8b4b-e68b-4161-92c1-1ae543b141b6 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:18.688697 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16488fa-58ac-4927-b225-f71ee1cf209f None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:18.708431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf8b4b-e68b-4161-92c1-1ae543b141b6 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:18.730721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:18.749568 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf1d04-ff00-482a-aa89-2bea88ff5afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:18.750220 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf1d04-ff00-482a-aa89-2bea88ff5afc None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:18.750496 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf1d04-ff00-482a-aa89-2bea88ff5afc None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:18.750868 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf1d04-ff00-482a-aa89-2bea88ff5afc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:18.767129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf1d04-ff00-482a-aa89-2bea88ff5afc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:18.768761 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf1d04-ff00-482a-aa89-2bea88ff5afc None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:18.771341 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdffe9-93ac-4ea4-ae9c-67d2a7d96a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:18.772325 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdffe9-93ac-4ea4-ae9c-67d2a7d96a56 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:18.772839 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdffe9-93ac-4ea4-ae9c-67d2a7d96a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:18.774466 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdffe9-93ac-4ea4-ae9c-67d2a7d96a56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:18.784664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-7fcf1d04-ff00-482a-aa89-2bea88ff5afc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:18.784664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:18.784664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:18.784664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:18.784664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:18.784664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:18.784664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:18.784664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:18.784664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:18.784664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:18.784664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:18.784664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:18.784664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:18.784664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:18.784664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:18.784664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:18.785706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:18.785706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:18.785706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:18.785706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:18.785706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:18.785706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:18.785706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:18.785706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:18.785706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:18.785706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:18.785706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:18.785706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:18.785706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:18.785706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:18.785706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:18.785706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:18.785706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:18.786686 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:18.786686 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:18.786686 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:18.791888 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 319/654] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:18.792608 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf1d04-ff00-482a-aa89-2bea88ff5afc None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:18.793805 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cdffe9-93ac-4ea4-ae9c-67d2a7d96a56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:18.795334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cdffe9-93ac-4ea4-ae9c-67d2a7d96a56 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:18.818256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-f9cdffe9-93ac-4ea4-ae9c-67d2a7d96a56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:18.818256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:18.818256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:18.818256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:18.818256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:18.818256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:18.818256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:18.818256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:18.818256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:18.818256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:18.818256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:18.818256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:18.818256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:18.818256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:18.818256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:18.818256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:18.819317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:18.819317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:18.819317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:18.819317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:18.819317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:18.819317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:18.819317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:18.819317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:18.819317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:18.819317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:18.819317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:18.819317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:18.819317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:18.819317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:18.819317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:18.819317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:18.819317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:18.820351 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:18.820351 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:18.820351 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:18.820351 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 336/655] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:18.821960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdffe9-93ac-4ea4-ae9c-67d2a7d96a56 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:18.846408 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:18.928219 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf1d04-ff00-482a-aa89-2bea88ff5afc None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:18.958806 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdffe9-93ac-4ea4-ae9c-67d2a7d96a56 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:18.969645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203da65-fc81-41e6-bb99-b4f3ec839b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:18.970753 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203da65-fc81-41e6-bb99-b4f3ec839b89 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:18.971099 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203da65-fc81-41e6-bb99-b4f3ec839b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:18.971435 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203da65-fc81-41e6-bb99-b4f3ec839b89 None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:18.972612 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203da65-fc81-41e6-bb99-b4f3ec839b89 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:18.973211 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203da65-fc81-41e6-bb99-b4f3ec839b89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:18.974702 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203da65-fc81-41e6-bb99-b4f3ec839b89 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:19.006791 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 320/656] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Aug 8 16:32:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:19.009090 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b752e-6c20-4c44-acd9-30717431107f None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:19.009582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b752e-6c20-4c44-acd9-30717431107f None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:19.010561 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b752e-6c20-4c44-acd9-30717431107f None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:19.017894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96285bea-59bc-47a7-ab3f-f265ebd30597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:19.018620 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96285bea-59bc-47a7-ab3f-f265ebd30597 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:19.018875 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96285bea-59bc-47a7-ab3f-f265ebd30597 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:19.019345 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96285bea-59bc-47a7-ab3f-f265ebd30597 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:19.019459 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 321/657] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:19 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:19.019814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96285bea-59bc-47a7-ab3f-f265ebd30597 None admin] RBAC: Adding query filter params (role.id=7764573472d34ecc8bedf7a9c7935170, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=9daa7489bced467984d39e8b8ac49d13) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:19.020158 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96285bea-59bc-47a7-ab3f-f265ebd30597 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:19.020891 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b752e-6c20-4c44-acd9-30717431107f None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:19.021624 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96285bea-59bc-47a7-ab3f-f265ebd30597 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:19.030138 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 337/658] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:32:18 2022] GET /identity/v3/role_assignments?user.id=9daa7489bced467984d39e8b8ac49d13&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=7764573472d34ecc8bedf7a9c7935170 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:19.031001 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96285bea-59bc-47a7-ab3f-f265ebd30597 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:19.101793 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:19.207868 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:19.220747 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b752e-6c20-4c44-acd9-30717431107f None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:19.223540 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96285bea-59bc-47a7-ab3f-f265ebd30597 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:19.286638 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97242a-eff2-484b-a6cc-6f755f2f1c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:19.287037 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97242a-eff2-484b-a6cc-6f755f2f1c1a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:19.287311 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97242a-eff2-484b-a6cc-6f755f2f1c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:19.287577 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97242a-eff2-484b-a6cc-6f755f2f1c1a None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:19.288267 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97242a-eff2-484b-a6cc-6f755f2f1c1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:19.288567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97242a-eff2-484b-a6cc-6f755f2f1c1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:19.290696 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97242a-eff2-484b-a6cc-6f755f2f1c1a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:19.296855 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157fcb6-c702-4b56-92af-0f3a17be9a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:19.297526 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157fcb6-c702-4b56-92af-0f3a17be9a1b None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:19.297765 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157fcb6-c702-4b56-92af-0f3a17be9a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:19.298069 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157fcb6-c702-4b56-92af-0f3a17be9a1b None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:19.299235 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3157fcb6-c702-4b56-92af-0f3a17be9a1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:19.299235 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3157fcb6-c702-4b56-92af-0f3a17be9a1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:19.300750 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3157fcb6-c702-4b56-92af-0f3a17be9a1b None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:19.312891 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 322/659] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:19.312891 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97242a-eff2-484b-a6cc-6f755f2f1c1a None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:19.317338 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 338/660] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:19.318803 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa17030-823c-420a-8c16-e9e47d13aee1 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:19.319093 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa17030-823c-420a-8c16-e9e47d13aee1 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:19.319346 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa17030-823c-420a-8c16-e9e47d13aee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:19.344916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-6aa17030-823c-420a-8c16-e9e47d13aee1 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:19.346814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-6aa17030-823c-420a-8c16-e9e47d13aee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:19.457022 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 339/661] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:32:19.458135 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa17030-823c-420a-8c16-e9e47d13aee1 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:19.488107 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97242a-eff2-484b-a6cc-6f755f2f1c1a None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:19.504893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:19.535046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54fc94-af07-42ce-8612-8fbbfe339ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:19.535823 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54fc94-af07-42ce-8612-8fbbfe339ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:19.536199 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54fc94-af07-42ce-8612-8fbbfe339ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:19.536561 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54fc94-af07-42ce-8612-8fbbfe339ab1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:19.581447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:19.618183 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa17030-823c-420a-8c16-e9e47d13aee1 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:19.660839 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54fc94-af07-42ce-8612-8fbbfe339ab1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:19.663016 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54fc94-af07-42ce-8612-8fbbfe339ab1 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:19.666203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b457a2-2cea-4f15-884f-e2e63d326699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:19.667000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b457a2-2cea-4f15-884f-e2e63d326699 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:19.667273 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b457a2-2cea-4f15-884f-e2e63d326699 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:19.667611 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b457a2-2cea-4f15-884f-e2e63d326699 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:19.675943 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b457a2-2cea-4f15-884f-e2e63d326699 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:19.678062 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b457a2-2cea-4f15-884f-e2e63d326699 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:19.690773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-07b457a2-2cea-4f15-884f-e2e63d326699 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:19.690773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:19.690773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:19.690773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:19.690773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:19.690773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:19.690773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:19.690773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:19.690773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:19.690773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:19.690773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:19.690773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:19.690773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:19.690773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:19.690773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:19.691763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:19.691763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:19.691763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:19.691763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:19.691763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:19.691763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:19.691763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:19.691763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:19.691763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:19.691763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:19.691763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:19.691763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:19.691763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:19.691763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:19.691763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:19.691763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:19.691763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:19.692694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:19.692694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:19.692694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:19.692694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:19.692694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:19.692694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:19.692694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:19.692694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:19.697361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 340/662] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:19.698238 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b457a2-2cea-4f15-884f-e2e63d326699 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:19.792963 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 323/663] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:32:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:32:19.794208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54fc94-af07-42ce-8612-8fbbfe339ab1 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:19.814899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:19.840252 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b457a2-2cea-4f15-884f-e2e63d326699 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:19.863087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:19.889022 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a89fa-862b-4a51-92ee-b20516019d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:19.890448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a89fa-862b-4a51-92ee-b20516019d84 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:19.890924 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a89fa-862b-4a51-92ee-b20516019d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:19.893462 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a89fa-862b-4a51-92ee-b20516019d84 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:19.904810 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a89fa-862b-4a51-92ee-b20516019d84 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:19.906518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a89fa-862b-4a51-92ee-b20516019d84 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:19.928884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-252a89fa-862b-4a51-92ee-b20516019d84 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 16:32:19.928884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:19.928884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:19.928884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:19.928884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:19.928884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:19.928884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:19.928884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:19.928884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:19.928884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:19.928884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:19.928884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:19.928884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:19.928884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:19.928884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:19.928884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:19.930264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:19.930264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:19.930264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:19.930264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:19.930264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:19.930264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:19.930264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:19.930264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:19.930264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:19.930264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:19.930264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:19.930264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:19.930264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:19.930264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:19.930264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:19.930264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:19.930264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:19.930264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:19.932030 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:19.932030 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:19.932030 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:19.932030 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:19.932030 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:19.932030 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:19.932030 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:19.932030 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:19.932030 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 16:32:19.932030 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:19.932614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54fc94-af07-42ce-8612-8fbbfe339ab1 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:19.934869 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 341/664] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:32:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:19.938827 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a89fa-862b-4a51-92ee-b20516019d84 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:19.992013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674dab73-52f8-404a-9c97-50957f8962ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:19.992660 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674dab73-52f8-404a-9c97-50957f8962ec None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:19.992940 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674dab73-52f8-404a-9c97-50957f8962ec None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:19.993272 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674dab73-52f8-404a-9c97-50957f8962ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:20.009435 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674dab73-52f8-404a-9c97-50957f8962ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:32:20.015582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-674dab73-52f8-404a-9c97-50957f8962ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:20.015582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:20.015582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:20.015582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:20.015582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:20.015582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:20.015582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:20.015582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:20.015582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:20.015582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:20.015582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:20.015582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:20.015582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:32:20.015582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:32:20.015582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:32:20.015582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:32:20.016529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:32:20.016529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:32:20.016529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:32:20.016529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:32:20.016529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:20.016529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:20.016529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:20.016529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:20.016529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:20.016529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:20.016529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:20.016529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:20.016529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:20.016529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:20.016529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:20.016529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:20.017396 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:20.017396 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:20.017396 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:20.017396 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:20.017396 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:32:20.017396 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:32:20.017396 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:32:20.017396 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:32:20.017396 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:32:20.017396 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:32:20.017396 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:20.017396 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:20.022275 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 324/665] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:32:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:20.023442 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674dab73-52f8-404a-9c97-50957f8962ec None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:20.076409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a89fa-862b-4a51-92ee-b20516019d84 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:20.113292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5b009-5223-4010-96e3-c0bfb8db4b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:20.114344 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5b009-5223-4010-96e3-c0bfb8db4b86 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:20.114917 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5b009-5223-4010-96e3-c0bfb8db4b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:20.115385 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5b009-5223-4010-96e3-c0bfb8db4b86 None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:20.116306 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e5b009-5223-4010-96e3-c0bfb8db4b86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:20.116942 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e5b009-5223-4010-96e3-c0bfb8db4b86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:20.119529 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e5b009-5223-4010-96e3-c0bfb8db4b86 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:20.132060 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:20.141492 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 342/666] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:20.146307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5b009-5223-4010-96e3-c0bfb8db4b86 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:20.146422 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674dab73-52f8-404a-9c97-50957f8962ec None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:20.160027 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:20.185157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10182f4-9ce8-41ac-8fc0-38b4385c1bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:20.185799 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10182f4-9ce8-41ac-8fc0-38b4385c1bde None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:20.186074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10182f4-9ce8-41ac-8fc0-38b4385c1bde None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:20.186338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10182f4-9ce8-41ac-8fc0-38b4385c1bde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:20.200588 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10182f4-9ce8-41ac-8fc0-38b4385c1bde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:20.202460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10182f4-9ce8-41ac-8fc0-38b4385c1bde None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:20.225767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-e10182f4-9ce8-41ac-8fc0-38b4385c1bde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:20.225767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:20.225767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:20.225767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:20.225767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:20.225767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:20.225767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:20.225767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:20.225767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:20.225767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:20.225767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:20.225767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:20.225767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:20.225767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:20.225767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:20.225767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:20.227015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:20.227015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:20.227015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:20.227015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:20.227015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:20.227015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:20.227015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:20.227015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:20.227015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:20.227015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:20.227015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:20.227015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:20.227015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:20.227015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:20.227015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:20.227015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:20.227015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:20.228105 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:20.228105 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:20.228105 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:20.229378 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 325/667] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:20.230208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10182f4-9ce8-41ac-8fc0-38b4385c1bde None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:20.333830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5b009-5223-4010-96e3-c0bfb8db4b86 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:20.388590 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10182f4-9ce8-41ac-8fc0-38b4385c1bde None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:20.390292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe180c-d49e-4ba8-87c7-2c098da53f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:20.391714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe180c-d49e-4ba8-87c7-2c098da53f5b None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:20.391714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe180c-d49e-4ba8-87c7-2c098da53f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:20.391909 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe180c-d49e-4ba8-87c7-2c098da53f5b None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:20.394172 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe180c-d49e-4ba8-87c7-2c098da53f5b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:20.394866 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe180c-d49e-4ba8-87c7-2c098da53f5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:20.398160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe180c-d49e-4ba8-87c7-2c098da53f5b None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:20.427779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 343/668] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Aug 8 16:32:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:20.429850 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe180c-d49e-4ba8-87c7-2c098da53f5b None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:20.443547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543d43e-70a7-4d79-a581-f6bcdf93cef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:20.446836 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543d43e-70a7-4d79-a581-f6bcdf93cef5 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:20.446836 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543d43e-70a7-4d79-a581-f6bcdf93cef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:20.446836 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543d43e-70a7-4d79-a581-f6bcdf93cef5 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:20.446836 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543d43e-70a7-4d79-a581-f6bcdf93cef5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:20.446836 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543d43e-70a7-4d79-a581-f6bcdf93cef5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:20.451191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543d43e-70a7-4d79-a581-f6bcdf93cef5 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:20.460906 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 326/669] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Aug 8 16:32:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:20.462515 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543d43e-70a7-4d79-a581-f6bcdf93cef5 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:20.495357 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:20.533799 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:20.591219 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe180c-d49e-4ba8-87c7-2c098da53f5b None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:20.640812 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543d43e-70a7-4d79-a581-f6bcdf93cef5 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:20.667831 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952c2c0-d005-4b39-b65c-47e30bfac464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:20.670731 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952c2c0-d005-4b39-b65c-47e30bfac464 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:20.672570 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952c2c0-d005-4b39-b65c-47e30bfac464 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:20.673769 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952c2c0-d005-4b39-b65c-47e30bfac464 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:20.676857 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2952c2c0-d005-4b39-b65c-47e30bfac464 None admin] RBAC: Adding query filter params (role.id=7764573472d34ecc8bedf7a9c7935170, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=9482bf89c1b648c1a91ee00ee5fbba63) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:20.678043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2952c2c0-d005-4b39-b65c-47e30bfac464 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:20.678594 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22d5ca-38e3-419c-9cbe-1c0faa56c0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:20.679232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22d5ca-38e3-419c-9cbe-1c0faa56c0ad None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:20.679511 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22d5ca-38e3-419c-9cbe-1c0faa56c0ad None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:20.679772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22d5ca-38e3-419c-9cbe-1c0faa56c0ad None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:20.680405 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22d5ca-38e3-419c-9cbe-1c0faa56c0ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:20.681280 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22d5ca-38e3-419c-9cbe-1c0faa56c0ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:20.681669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2952c2c0-d005-4b39-b65c-47e30bfac464 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:20.682623 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22d5ca-38e3-419c-9cbe-1c0faa56c0ad None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:20.691825 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 344/670] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:32:20 2022] GET /identity/v3/role_assignments?user.id=9482bf89c1b648c1a91ee00ee5fbba63&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=7764573472d34ecc8bedf7a9c7935170 => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:20.693069 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952c2c0-d005-4b39-b65c-47e30bfac464 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:20.706548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 327/671] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:20.707511 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22d5ca-38e3-419c-9cbe-1c0faa56c0ad None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:20.971916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:21.010510 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952c2c0-d005-4b39-b65c-47e30bfac464 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:21.026493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22d5ca-38e3-419c-9cbe-1c0faa56c0ad None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:21.088544 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:21.106124 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abade210-bafe-4026-a370-5c063ff19839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:21.107174 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abade210-bafe-4026-a370-5c063ff19839 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:21.107658 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abade210-bafe-4026-a370-5c063ff19839 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:21.108130 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abade210-bafe-4026-a370-5c063ff19839 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:21.136006 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2ee10-423d-4d47-b596-51ac8ef22ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:21.136645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2ee10-423d-4d47-b596-51ac8ef22ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:21.136918 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2ee10-423d-4d47-b596-51ac8ef22ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:21.137184 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2ee10-423d-4d47-b596-51ac8ef22ef1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:21.152749 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2ee10-423d-4d47-b596-51ac8ef22ef1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:21.154296 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2ee10-423d-4d47-b596-51ac8ef22ef1 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:21.173700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-72d2ee10-423d-4d47-b596-51ac8ef22ef1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:21.173700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:21.173700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:21.173700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:21.173700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:21.173700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:21.173700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:21.173700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:21.173700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:21.173700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:21.173700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:21.173700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:21.173700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:21.173700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:21.173700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:21.173700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:21.174819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:21.174819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:21.174819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:21.174819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:21.174819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:21.174819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:21.174819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:21.174819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:21.174819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:21.174819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:21.174819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:21.174819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:21.174819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:21.174819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:21.174819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:21.174819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:21.174819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:21.175843 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:21.175843 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:21.175843 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:21.178640 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 328/672] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 468 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:21.179789 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2ee10-423d-4d47-b596-51ac8ef22ef1 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:21.209199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abade210-bafe-4026-a370-5c063ff19839 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:21.210639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abade210-bafe-4026-a370-5c063ff19839 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:21.319210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2ee10-423d-4d47-b596-51ac8ef22ef1 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:21.329791 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 345/673] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:32:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:32:21.330973 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abade210-bafe-4026-a370-5c063ff19839 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:21.358032 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:21.376537 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a74eb3-f4b7-4f0e-97c4-8587c5c35232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:21.384328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a74eb3-f4b7-4f0e-97c4-8587c5c35232 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:21.384328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a74eb3-f4b7-4f0e-97c4-8587c5c35232 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:21.384328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a74eb3-f4b7-4f0e-97c4-8587c5c35232 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:21.388232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a74eb3-f4b7-4f0e-97c4-8587c5c35232 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:21.389895 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a74eb3-f4b7-4f0e-97c4-8587c5c35232 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:21.392699 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:21.403058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-73a74eb3-f4b7-4f0e-97c4-8587c5c35232 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:21.403058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:21.403058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:21.403058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:21.403058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:21.403058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:21.403058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:21.403058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:21.403058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:21.403058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:21.403058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:21.403058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:21.403058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:21.403058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:21.403058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:21.404018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:21.404018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:21.404018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:21.404018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:21.404018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:21.404018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:21.404018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:21.404018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:21.404018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:21.404018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:21.404018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:21.404018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:21.404018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:21.404018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:21.404018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:21.404018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:21.404018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:21.404937 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:21.404937 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:21.404937 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:21.404937 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:21.404937 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:21.404937 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:21.404937 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:21.404937 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:21.409081 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 329/674] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:21.410423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51308ef5-cd8d-43c6-bf88-026709eb0bad None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:21.410728 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51308ef5-cd8d-43c6-bf88-026709eb0bad None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:21.411000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51308ef5-cd8d-43c6-bf88-026709eb0bad None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:21.412764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 330/675] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:21.413282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51308ef5-cd8d-43c6-bf88-026709eb0bad None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:21.458517 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abade210-bafe-4026-a370-5c063ff19839 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:21.500744 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab8e7d-2a4c-48a9-a9e0-236b74856819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:21.501266 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab8e7d-2a4c-48a9-a9e0-236b74856819 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:21.501519 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab8e7d-2a4c-48a9-a9e0-236b74856819 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:21.501794 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab8e7d-2a4c-48a9-a9e0-236b74856819 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:21.503197 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab8e7d-2a4c-48a9-a9e0-236b74856819 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:21.503197 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab8e7d-2a4c-48a9-a9e0-236b74856819 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:21.504126 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab8e7d-2a4c-48a9-a9e0-236b74856819 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:21.519126 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 346/676] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:21.519825 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab8e7d-2a4c-48a9-a9e0-236b74856819 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:21.549984 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51308ef5-cd8d-43c6-bf88-026709eb0bad None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:21.598104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b11a6-ca0b-4b16-acba-71ce056f804a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:21.598761 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b11a6-ca0b-4b16-acba-71ce056f804a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:21.599040 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b11a6-ca0b-4b16-acba-71ce056f804a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:21.599341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b11a6-ca0b-4b16-acba-71ce056f804a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:21.611298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b11a6-ca0b-4b16-acba-71ce056f804a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:21.612793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b11a6-ca0b-4b16-acba-71ce056f804a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:21.628613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-2c6b11a6-ca0b-4b16-acba-71ce056f804a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:21.628613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:21.628613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:21.628613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:21.628613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:21.628613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:21.628613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:21.628613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:21.628613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:21.628613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:21.628613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:21.628613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:21.628613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:21.628613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:21.628613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:21.628613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:21.631098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:21.631098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:21.631098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:21.631098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:21.631098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:21.631098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:21.631098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:21.631098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:21.631098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:21.631098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:21.631098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:21.631098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:21.631098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:21.631098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:21.631098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:21.631098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:21.631098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:21.632633 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:21.632633 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:21.632633 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:21.632633 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 331/677] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:21.632633 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2db868-0302-41fc-8073-600d53adf65f None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:21.632633 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2db868-0302-41fc-8073-600d53adf65f None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:21.632633 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2db868-0302-41fc-8073-600d53adf65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:21.652672 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-4f2db868-0302-41fc-8073-600d53adf65f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:21.653991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-4f2db868-0302-41fc-8073-600d53adf65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:21.725953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:21.749071 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:21.758472 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 332/678] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:32:21.758472 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2db868-0302-41fc-8073-600d53adf65f None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:21.786672 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab8e7d-2a4c-48a9-a9e0-236b74856819 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:21.827865 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0b2d0-f3b5-4dc6-93a5-8947f03062f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:21.828840 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0b2d0-f3b5-4dc6-93a5-8947f03062f4 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:21.829256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0b2d0-f3b5-4dc6-93a5-8947f03062f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:21.829540 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0b2d0-f3b5-4dc6-93a5-8947f03062f4 None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:21.830253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0b2d0-f3b5-4dc6-93a5-8947f03062f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:21.830556 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0b2d0-f3b5-4dc6-93a5-8947f03062f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:21.832079 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0b2d0-f3b5-4dc6-93a5-8947f03062f4 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:21.849936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 347/679] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:21.849936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0b2d0-f3b5-4dc6-93a5-8947f03062f4 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:21.889151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2db868-0302-41fc-8073-600d53adf65f None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:21.938953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b0537-3cd8-438a-90d9-6e31b9710712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:21.939812 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b0537-3cd8-438a-90d9-6e31b9710712 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:21.940251 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b0537-3cd8-438a-90d9-6e31b9710712 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:21.940688 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b0537-3cd8-438a-90d9-6e31b9710712 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:21.955362 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b0537-3cd8-438a-90d9-6e31b9710712 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:21.957627 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b0537-3cd8-438a-90d9-6e31b9710712 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:21.983506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-437b0537-3cd8-438a-90d9-6e31b9710712 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:32:21.983506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:21.983506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:21.983506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:21.983506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:21.983506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:21.983506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:21.983506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:21.983506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:21.983506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:21.983506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:21.983506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:21.983506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:21.983506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:21.983506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:21.983506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:21.985299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:21.985299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:21.985299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:21.985299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:21.985299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:21.985299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:21.985299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:21.985299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:21.985299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:21.985299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:21.985299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:21.985299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:21.985299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:21.985299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:21.985299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:21.985299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:21.985299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:21.985299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:21.988038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:21.988038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:21.988038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:21.988038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:21.988038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:21.988038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:21.988038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:21.988038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:21.988038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:32:21.988038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:21.990039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0b2d0-f3b5-4dc6-93a5-8947f03062f4 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:21.990269 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 333/680] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:32:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:21.990269 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b0537-3cd8-438a-90d9-6e31b9710712 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:22.031916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf06ab-3d6c-4bad-bcae-11bb783f3f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:22.032468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf06ab-3d6c-4bad-bcae-11bb783f3f3b None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:22.032744 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf06ab-3d6c-4bad-bcae-11bb783f3f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:22.033017 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf06ab-3d6c-4bad-bcae-11bb783f3f3b None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:22.033662 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf06ab-3d6c-4bad-bcae-11bb783f3f3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:22.033965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf06ab-3d6c-4bad-bcae-11bb783f3f3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:22.035366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf06ab-3d6c-4bad-bcae-11bb783f3f3b None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:22.048218 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 348/681] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:22.049086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf06ab-3d6c-4bad-bcae-11bb783f3f3b None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:22.054958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:22.064044 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:22.166368 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b0537-3cd8-438a-90d9-6e31b9710712 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:22.241309 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98f6f8-f2f6-4db9-b2e3-0bdb07663a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:22.242084 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf06ab-3d6c-4bad-bcae-11bb783f3f3b None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:22.242293 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98f6f8-f2f6-4db9-b2e3-0bdb07663a7c None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:22.242588 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98f6f8-f2f6-4db9-b2e3-0bdb07663a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:22.244465 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98f6f8-f2f6-4db9-b2e3-0bdb07663a7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:22.259371 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca98f6f8-f2f6-4db9-b2e3-0bdb07663a7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:32:22.268002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-ca98f6f8-f2f6-4db9-b2e3-0bdb07663a7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:32:22.268002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:22.268002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:22.268002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:22.268002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:22.268002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:22.268002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:22.268002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:22.268002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:22.268002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:22.268002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:22.268002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:22.268002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:32:22.268002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:32:22.268002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:32:22.268002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:32:22.270186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:32:22.270186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:32:22.270186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:32:22.270186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:32:22.270186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:22.270186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:22.270186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:22.270186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:22.270186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:22.270186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:22.270186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:22.270186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:22.270186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:22.270186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:22.270186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:22.270186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:22.272379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:22.272379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:22.272379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:22.272379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:22.272379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:32:22.272379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:32:22.272379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:32:22.272379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:32:22.272379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:32:22.272379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:32:22.272379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:32:22.272379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:22.276695 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 334/682] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:32:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:22.276695 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98f6f8-f2f6-4db9-b2e3-0bdb07663a7c None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:22.319718 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d233c-2871-4b66-90d0-352927999282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:22.320629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d233c-2871-4b66-90d0-352927999282 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:22.322135 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d233c-2871-4b66-90d0-352927999282 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:22.322891 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d233c-2871-4b66-90d0-352927999282 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:22.324234 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d233c-2871-4b66-90d0-352927999282 None admin] RBAC: Adding query filter params (role.id=7764573472d34ecc8bedf7a9c7935170, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=ea51ef02128a4b63b11049eb0daef298) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:22.324765 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d233c-2871-4b66-90d0-352927999282 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:22.331805 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d233c-2871-4b66-90d0-352927999282 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:22.351155 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 349/683] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:32:22 2022] GET /identity/v3/role_assignments?user.id=ea51ef02128a4b63b11049eb0daef298&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=7764573472d34ecc8bedf7a9c7935170 => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:22.353199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d233c-2871-4b66-90d0-352927999282 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:22.429233 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98f6f8-f2f6-4db9-b2e3-0bdb07663a7c None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:22.447148 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:22.502155 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:22.543566 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3052ab-126d-4abc-b059-1558b8642aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:22.544198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3052ab-126d-4abc-b059-1558b8642aac None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:22.544481 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3052ab-126d-4abc-b059-1558b8642aac None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:22.544739 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3052ab-126d-4abc-b059-1558b8642aac None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:22.545500 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3052ab-126d-4abc-b059-1558b8642aac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:22.545800 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3052ab-126d-4abc-b059-1558b8642aac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:22.548591 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3052ab-126d-4abc-b059-1558b8642aac None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:22.591130 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d233c-2871-4b66-90d0-352927999282 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:22.593188 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 335/684] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Aug 8 16:32:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:22.594189 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3052ab-126d-4abc-b059-1558b8642aac None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:22.676799 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3e11e-e6a2-46c2-8c9d-0c17dcedf1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:22.677816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3e11e-e6a2-46c2-8c9d-0c17dcedf1be None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:22.678264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3e11e-e6a2-46c2-8c9d-0c17dcedf1be None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:22.678734 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3e11e-e6a2-46c2-8c9d-0c17dcedf1be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:22.690527 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3e11e-e6a2-46c2-8c9d-0c17dcedf1be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:22.692437 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3e11e-e6a2-46c2-8c9d-0c17dcedf1be None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:22.709446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-d3b3e11e-e6a2-46c2-8c9d-0c17dcedf1be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:22.709446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:22.709446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:22.709446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:22.709446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:22.709446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:22.709446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:22.709446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:22.709446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:22.709446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:22.709446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:22.709446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:22.709446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:22.709446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:22.709446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:22.711289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:22.711289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:22.711289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:22.711289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:22.711289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:22.711289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:22.711289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:22.711289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:22.711289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:22.711289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:22.711289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:22.711289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:22.711289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:22.711289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:22.711289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:22.711289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:22.711289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:22.713653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:22.713653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:22.713653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:22.713653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:22.713653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:22.713653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:22.713653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:22.713653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:22.713653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 350/685] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:22.716421 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3e11e-e6a2-46c2-8c9d-0c17dcedf1be None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:22.828562 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3052ab-126d-4abc-b059-1558b8642aac None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:22.894340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dfa4b9-5dbe-4b78-9192-ff73aace1015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:22.895177 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dfa4b9-5dbe-4b78-9192-ff73aace1015 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:22.895582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dfa4b9-5dbe-4b78-9192-ff73aace1015 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:22.896011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dfa4b9-5dbe-4b78-9192-ff73aace1015 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:22.896810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dfa4b9-5dbe-4b78-9192-ff73aace1015 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:22.897232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dfa4b9-5dbe-4b78-9192-ff73aace1015 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:22.898911 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dfa4b9-5dbe-4b78-9192-ff73aace1015 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:22.910201 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3e11e-e6a2-46c2-8c9d-0c17dcedf1be None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:22.914625 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 336/686] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:32:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:22.914625 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dfa4b9-5dbe-4b78-9192-ff73aace1015 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:22.944825 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:22.954339 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:22.956109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8300747-ff9e-4dde-b970-6f0f3ecf7d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:22.956877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8300747-ff9e-4dde-b970-6f0f3ecf7d3c None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:22.957980 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8300747-ff9e-4dde-b970-6f0f3ecf7d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:22.957980 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8300747-ff9e-4dde-b970-6f0f3ecf7d3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:23.040724 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8300747-ff9e-4dde-b970-6f0f3ecf7d3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:23.042361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8300747-ff9e-4dde-b970-6f0f3ecf7d3c None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:23.049857 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dfa4b9-5dbe-4b78-9192-ff73aace1015 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:23.103789 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75c834-9985-45f4-a36a-ea34437bb376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:23.104614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75c834-9985-45f4-a36a-ea34437bb376 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:23.105053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75c834-9985-45f4-a36a-ea34437bb376 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:23.105500 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75c834-9985-45f4-a36a-ea34437bb376 None admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:23.106592 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75c834-9985-45f4-a36a-ea34437bb376 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:23.107106 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75c834-9985-45f4-a36a-ea34437bb376 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:23.109542 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75c834-9985-45f4-a36a-ea34437bb376 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:23.127794 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 337/687] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:23.129068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4492762-68b3-4493-a8b8-7396fde66eea None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:23.129349 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4492762-68b3-4493-a8b8-7396fde66eea None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:23.129618 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4492762-68b3-4493-a8b8-7396fde66eea None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:23.134731 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 338/688] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:23.134731 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4492762-68b3-4493-a8b8-7396fde66eea None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:23.195649 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 351/689] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:32:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:32:23.195649 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed964eae-36d1-4998-b7be-f33c3519cfbc None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:23.195649 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed964eae-36d1-4998-b7be-f33c3519cfbc None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:23.195649 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed964eae-36d1-4998-b7be-f33c3519cfbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:23.212948 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-ed964eae-36d1-4998-b7be-f33c3519cfbc None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:23.214446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ed964eae-36d1-4998-b7be-f33c3519cfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:23.245865 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:23.253483 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:23.273183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4492762-68b3-4493-a8b8-7396fde66eea None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:23.312370 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5a42d-77fd-4270-bb63-3b1950f6a3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:23.313008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5a42d-77fd-4270-bb63-3b1950f6a3c3 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:23.313279 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5a42d-77fd-4270-bb63-3b1950f6a3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:23.313544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5a42d-77fd-4270-bb63-3b1950f6a3c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:23.324696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 352/690] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:32:23.325589 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed964eae-36d1-4998-b7be-f33c3519cfbc None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:23.335044 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e5a42d-77fd-4270-bb63-3b1950f6a3c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:23.338548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e5a42d-77fd-4270-bb63-3b1950f6a3c3 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:23.367312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-a7e5a42d-77fd-4270-bb63-3b1950f6a3c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:23.367312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:23.367312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:23.367312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:23.367312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:23.367312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:23.367312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:23.367312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:23.367312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:23.367312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:23.367312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:23.367312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:23.367312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:23.367312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:23.367312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:23.367312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:23.369679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:23.369679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:23.369679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:23.369679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:23.369679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:23.369679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:23.369679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:23.369679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:23.369679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:23.369679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:23.369679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:23.369679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:23.369679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:23.369679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:23.369679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:23.369679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:23.369679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:23.371352 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:23.371352 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:23.371352 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:23.384769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 339/691] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:23.384769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5a42d-77fd-4270-bb63-3b1950f6a3c3 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:23.465747 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed964eae-36d1-4998-b7be-f33c3519cfbc None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:23.506854 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f9788-910a-4f75-82ce-5698b2c4fd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:23.507883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f9788-910a-4f75-82ce-5698b2c4fd81 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:23.507883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f9788-910a-4f75-82ce-5698b2c4fd81 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:23.508184 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f9788-910a-4f75-82ce-5698b2c4fd81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:23.508726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f9788-910a-4f75-82ce-5698b2c4fd81 None admin] RBAC: Adding query filter params (role.id=7764573472d34ecc8bedf7a9c7935170, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=7bbf0678f34e46e8b56aaeabce955180) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:23.509201 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f9788-910a-4f75-82ce-5698b2c4fd81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:23.511025 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f9788-910a-4f75-82ce-5698b2c4fd81 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:23.514017 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5a42d-77fd-4270-bb63-3b1950f6a3c3 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:23.517535 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 353/692] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:32:23 2022] GET /identity/v3/role_assignments?user.id=7bbf0678f34e46e8b56aaeabce955180&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=7764573472d34ecc8bedf7a9c7935170 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:23.524394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f9788-910a-4f75-82ce-5698b2c4fd81 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:23.582383 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0c9ab-0824-4b36-981a-a05b3dbd5186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:23.584304 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:23.585014 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0c9ab-0824-4b36-981a-a05b3dbd5186 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:23.585423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0c9ab-0824-4b36-981a-a05b3dbd5186 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:23.586554 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0c9ab-0824-4b36-981a-a05b3dbd5186 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:23.590918 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:23.612494 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec0c9ab-0824-4b36-981a-a05b3dbd5186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:23.616220 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec0c9ab-0824-4b36-981a-a05b3dbd5186 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:23.655140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-7ec0c9ab-0824-4b36-981a-a05b3dbd5186 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:23.655140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:23.655140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:23.655140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:23.655140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:23.655140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:23.655140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:23.655140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:23.655140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:23.655140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:23.655140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:23.655140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:23.655140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:23.655140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:23.655140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:23.655140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:23.657153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:23.657153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:23.657153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:23.657153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:23.657153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:23.657153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:23.657153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:23.657153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:23.657153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:23.657153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:23.657153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:23.657153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:23.657153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:23.657153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:23.657153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:23.657153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:23.657153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:23.659152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:23.659152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:23.659152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:23.668452 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 340/693] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:23.668452 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0c9ab-0824-4b36-981a-a05b3dbd5186 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:23.779664 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f9788-910a-4f75-82ce-5698b2c4fd81 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:23.828629 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0c9ab-0824-4b36-981a-a05b3dbd5186 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:23.834227 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656cd69-c912-4c21-8c10-59337557ad2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:23.835209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656cd69-c912-4c21-8c10-59337557ad2c None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:23.835639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656cd69-c912-4c21-8c10-59337557ad2c None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:23.836081 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656cd69-c912-4c21-8c10-59337557ad2c None admin] PATH_INFO: `/v3/services/network` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:23.837397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7656cd69-c912-4c21-8c10-59337557ad2c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:23.839031 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7656cd69-c912-4c21-8c10-59337557ad2c None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:23.853590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-7656cd69-c912-4c21-8c10-59337557ad2c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 16:32:23.853590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:23.853590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:32:23.853590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:32:23.853590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:32:23.853590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:32:23.853590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:32:23.853590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:23.853590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 16:32:23.853590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:23.853590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:32:23.853590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:23.853590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:23.853590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:23.853590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:23.853590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:23.853590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:23.854899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:23.854899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:23.854899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:23.854899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:23.854899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:23.854899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:23.854899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:32:23.854899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:32:23.854899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:32:23.854899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:32:23.854899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:23.854899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:23.854899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:23.854899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:23.854899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:23.854899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:23.855880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:23.855880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:23.855880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:23.855880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:23.855880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:23.855880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:23.855880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:23.855880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:23.855880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:23.855880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:23.855880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:32:23.855880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:23.855880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 16:32:23.855880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:23.858416 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 354/694] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:23 2022] GET /identity/v3/services/network => generated 88 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:23.859410 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656cd69-c912-4c21-8c10-59337557ad2c None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:23.907172 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea82be-f9af-4bf6-aea7-31036e20fdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:23.908429 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea82be-f9af-4bf6-aea7-31036e20fdc8 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:23.909024 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea82be-f9af-4bf6-aea7-31036e20fdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:23.909619 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea82be-f9af-4bf6-aea7-31036e20fdc8 None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:23.910925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea82be-f9af-4bf6-aea7-31036e20fdc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:23.911609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea82be-f9af-4bf6-aea7-31036e20fdc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:23.914647 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea82be-f9af-4bf6-aea7-31036e20fdc8 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:23.951152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 341/695] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:23.952758 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea82be-f9af-4bf6-aea7-31036e20fdc8 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:23.968366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:23.988513 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656cd69-c912-4c21-8c10-59337557ad2c None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:24.032642 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b419b7-ca46-480e-aa6a-c9357c9dd76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:24.033503 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b419b7-ca46-480e-aa6a-c9357c9dd76d None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:24.033949 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b419b7-ca46-480e-aa6a-c9357c9dd76d None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:24.034363 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b419b7-ca46-480e-aa6a-c9357c9dd76d None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:24.035253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b419b7-ca46-480e-aa6a-c9357c9dd76d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:24.035699 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b419b7-ca46-480e-aa6a-c9357c9dd76d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:24.037239 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b419b7-ca46-480e-aa6a-c9357c9dd76d None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:24.051687 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:24.054543 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 355/696] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:24.055619 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b419b7-ca46-480e-aa6a-c9357c9dd76d None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:24.132845 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea82be-f9af-4bf6-aea7-31036e20fdc8 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:24.176579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f598bb-c851-44a9-984d-1c6c15e4898e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:24.177448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f598bb-c851-44a9-984d-1c6c15e4898e None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:24.177904 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f598bb-c851-44a9-984d-1c6c15e4898e None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:24.178316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f598bb-c851-44a9-984d-1c6c15e4898e None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:24.179132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f598bb-c851-44a9-984d-1c6c15e4898e None admin] RBAC: Adding query filter params (name=network) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:24.179579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f598bb-c851-44a9-984d-1c6c15e4898e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:24.181174 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f598bb-c851-44a9-984d-1c6c15e4898e None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:24.187501 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 342/697] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Aug 8 16:32:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:24.188549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f598bb-c851-44a9-984d-1c6c15e4898e None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:24.244911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b419b7-ca46-480e-aa6a-c9357c9dd76d None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:24.306828 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800c73b-e392-4726-90f3-b6b6399f2c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:24.307370 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800c73b-e392-4726-90f3-b6b6399f2c37 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:24.307635 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800c73b-e392-4726-90f3-b6b6399f2c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:24.307942 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800c73b-e392-4726-90f3-b6b6399f2c37 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:24.317726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4800c73b-e392-4726-90f3-b6b6399f2c37 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:24.319218 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4800c73b-e392-4726-90f3-b6b6399f2c37 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:24.331021 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f598bb-c851-44a9-984d-1c6c15e4898e None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:24.331887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-4800c73b-e392-4726-90f3-b6b6399f2c37 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:32:24.331887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:24.331887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:24.331887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:24.331887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:24.331887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:24.331887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:24.331887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:24.331887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:24.331887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:24.331887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:24.331887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:24.331887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:24.331887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:24.331887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:24.331887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:24.333894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:24.333894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:24.333894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:24.333894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:24.333894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:24.333894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:24.333894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:24.333894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:24.333894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:24.333894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:24.333894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:24.333894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:24.333894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:24.333894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:24.333894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:24.333894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:24.333894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:24.333894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:24.336129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:24.336129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:24.336129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:24.336129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:24.336129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:24.336129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:24.336129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:24.336129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:24.336129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:32:24.336129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:24.338748 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 356/698] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Aug 8 16:32:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:24.339190 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800c73b-e392-4726-90f3-b6b6399f2c37 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:24.343500 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:24.364117 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:24.413248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03639b30-4f90-4ae3-abea-db1404bcda27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:24.416208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03639b30-4f90-4ae3-abea-db1404bcda27 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:24.417004 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03639b30-4f90-4ae3-abea-db1404bcda27 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:24.417680 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03639b30-4f90-4ae3-abea-db1404bcda27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:24.432103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03639b30-4f90-4ae3-abea-db1404bcda27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:24.435286 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03639b30-4f90-4ae3-abea-db1404bcda27 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:24.453601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800c73b-e392-4726-90f3-b6b6399f2c37 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:24.463509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-03639b30-4f90-4ae3-abea-db1404bcda27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:24.463509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:24.463509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:24.463509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:24.463509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:24.463509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:24.463509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:24.463509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:24.463509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:24.463509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:24.463509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:24.463509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:24.463509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:24.463509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:24.463509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:24.465273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:24.465273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:24.465273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:24.465273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:24.465273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:24.465273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:24.465273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:24.465273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:24.465273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:24.465273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:24.465273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:24.465273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:24.465273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:24.465273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:24.465273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:24.465273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:24.465273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:24.467058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:24.467058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:24.467058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:24.467058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:24.467058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:24.467058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:24.467058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:24.467058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:24.475198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 343/699] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:24.475198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03639b30-4f90-4ae3-abea-db1404bcda27 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:24.514978 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76424a02-8728-431b-9633-3f6a2d3c9e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:24.517757 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76424a02-8728-431b-9633-3f6a2d3c9e7b None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:24.519182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76424a02-8728-431b-9633-3f6a2d3c9e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:24.520349 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76424a02-8728-431b-9633-3f6a2d3c9e7b None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:24.522724 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76424a02-8728-431b-9633-3f6a2d3c9e7b None admin] RBAC: Adding query filter params (type=network) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:24.523921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76424a02-8728-431b-9633-3f6a2d3c9e7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:24.528435 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76424a02-8728-431b-9633-3f6a2d3c9e7b None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:24.551407 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 357/700] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Aug 8 16:32:24 2022] GET /identity/v3/services?type=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:24.551407 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76424a02-8728-431b-9633-3f6a2d3c9e7b None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:24.672689 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03639b30-4f90-4ae3-abea-db1404bcda27 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:24.729599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:24.764988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006eac0-7ff0-47bf-aa06-a835da4611d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:24.764988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006eac0-7ff0-47bf-aa06-a835da4611d8 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:24.764988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006eac0-7ff0-47bf-aa06-a835da4611d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:24.765533 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006eac0-7ff0-47bf-aa06-a835da4611d8 None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:24.768179 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006eac0-7ff0-47bf-aa06-a835da4611d8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:24.768526 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006eac0-7ff0-47bf-aa06-a835da4611d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:24.769963 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006eac0-7ff0-47bf-aa06-a835da4611d8 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:24.809091 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 344/701] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Aug 8 16:32:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:24.815523 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006eac0-7ff0-47bf-aa06-a835da4611d8 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:24.818118 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76424a02-8728-431b-9633-3f6a2d3c9e7b None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:24.881992 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:24.888454 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea17e8-40a5-4517-9fd9-55c68c7181c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:24.889082 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea17e8-40a5-4517-9fd9-55c68c7181c2 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:24.889356 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea17e8-40a5-4517-9fd9-55c68c7181c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:24.889626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea17e8-40a5-4517-9fd9-55c68c7181c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:24.893586 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea17e8-40a5-4517-9fd9-55c68c7181c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:24.893586 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea17e8-40a5-4517-9fd9-55c68c7181c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:24.893586 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea17e8-40a5-4517-9fd9-55c68c7181c2 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:24.916320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 358/702] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:24.938234 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea17e8-40a5-4517-9fd9-55c68c7181c2 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:25.057453 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006eac0-7ff0-47bf-aa06-a835da4611d8 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:25.087274 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea17e8-40a5-4517-9fd9-55c68c7181c2 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:25.143616 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:25.160240 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef02b34-68ba-4c2e-843f-84dab5163921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:25.162262 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef02b34-68ba-4c2e-843f-84dab5163921 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:25.162783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef02b34-68ba-4c2e-843f-84dab5163921 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:25.163266 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef02b34-68ba-4c2e-843f-84dab5163921 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:25.164728 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef02b34-68ba-4c2e-843f-84dab5163921 None admin] RBAC: Adding query filter params (role.id=7764573472d34ecc8bedf7a9c7935170, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=c606decb832444e38a194b98f8d15f8c) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:25.165193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef02b34-68ba-4c2e-843f-84dab5163921 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:25.166969 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a1168-7ee0-4772-a1cd-b44e2d533f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:25.167671 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a1168-7ee0-4772-a1cd-b44e2d533f9a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:25.167972 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef02b34-68ba-4c2e-843f-84dab5163921 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:25.168136 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a1168-7ee0-4772-a1cd-b44e2d533f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:25.168388 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a1168-7ee0-4772-a1cd-b44e2d533f9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:25.183349 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 359/703] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:32:24 2022] GET /identity/v3/role_assignments?user.id=c606decb832444e38a194b98f8d15f8c&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=7764573472d34ecc8bedf7a9c7935170 => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:32:25.186339 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3f032-9689-4822-8306-90d9ed1192ea None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:25.187420 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3f032-9689-4822-8306-90d9ed1192ea None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:25.188127 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3f032-9689-4822-8306-90d9ed1192ea None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:25.191646 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 360/704] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:25.200367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f666a70-c3ad-4a51-abdd-b839d1f1a78b None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:25.200787 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f666a70-c3ad-4a51-abdd-b839d1f1a78b None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:25.201304 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f666a70-c3ad-4a51-abdd-b839d1f1a78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:25.225143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-3f666a70-c3ad-4a51-abdd-b839d1f1a78b None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:25.226508 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-3f666a70-c3ad-4a51-abdd-b839d1f1a78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:25.287102 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a1168-7ee0-4772-a1cd-b44e2d533f9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:25.288778 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a1168-7ee0-4772-a1cd-b44e2d533f9a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:25.327015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:25.345396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 361/705] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:32:25.370848 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f666a70-c3ad-4a51-abdd-b839d1f1a78b None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:25.493940 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 345/706] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:32:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:32:25.495804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a1168-7ee0-4772-a1cd-b44e2d533f9a None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:25.575487 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f666a70-c3ad-4a51-abdd-b839d1f1a78b None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:25.604083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:25.618791 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1369dc1-927c-423d-9001-1ebb21e4c4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:25.619621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1369dc1-927c-423d-9001-1ebb21e4c4fe None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:25.620211 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1369dc1-927c-423d-9001-1ebb21e4c4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:25.620469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1369dc1-927c-423d-9001-1ebb21e4c4fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:25.633164 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1369dc1-927c-423d-9001-1ebb21e4c4fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:25.635477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1369dc1-927c-423d-9001-1ebb21e4c4fe None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:25.646180 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a1168-7ee0-4772-a1cd-b44e2d533f9a None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:25.653087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-e1369dc1-927c-423d-9001-1ebb21e4c4fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:25.653087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:25.653087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:25.653087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:25.653087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:25.653087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:25.653087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:25.653087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:25.653087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:25.653087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:25.653087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:25.653087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:25.653087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:25.653087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:25.653087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:25.653087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:25.654221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:25.654221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:25.654221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:25.654221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:25.654221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:25.654221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:25.654221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:25.654221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:25.654221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:25.654221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:25.654221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:25.654221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:25.654221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:25.654221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:25.654221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:25.654221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:25.654221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:25.655218 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:25.655218 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:25.655218 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:25.656334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 362/707] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:32:25.667796 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1369dc1-927c-423d-9001-1ebb21e4c4fe None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:25.680749 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:25.690110 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f5c6b-374a-4d9c-a936-c70a5f065823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:25.690405 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f5c6b-374a-4d9c-a936-c70a5f065823 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:25.690914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f5c6b-374a-4d9c-a936-c70a5f065823 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:25.691179 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f5c6b-374a-4d9c-a936-c70a5f065823 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:25.705476 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f5c6b-374a-4d9c-a936-c70a5f065823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:25.706999 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f5c6b-374a-4d9c-a936-c70a5f065823 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:25.723953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-636f5c6b-374a-4d9c-a936-c70a5f065823 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:25.723953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:25.723953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:25.723953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:25.723953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:25.723953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:25.723953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:25.723953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:25.723953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:25.723953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:25.723953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:25.723953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:25.723953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:25.723953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:25.723953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:25.723953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:25.725008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:25.725008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:25.725008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:25.725008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:25.725008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:25.725008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:25.725008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:25.725008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:25.725008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:25.725008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:25.725008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:25.725008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:25.725008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:25.725008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:25.725008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:25.725008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:25.725008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:25.725922 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:25.725922 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:25.725922 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:25.725922 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 346/708] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:32:25.732910 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f5c6b-374a-4d9c-a936-c70a5f065823 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:25.861910 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f5c6b-374a-4d9c-a936-c70a5f065823 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:25.916677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3fa2a-6154-401c-a338-c717f390de12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:25.917576 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3fa2a-6154-401c-a338-c717f390de12 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:25.918173 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3fa2a-6154-401c-a338-c717f390de12 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:25.918452 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3fa2a-6154-401c-a338-c717f390de12 None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:25.921908 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3fa2a-6154-401c-a338-c717f390de12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:25.921908 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3fa2a-6154-401c-a338-c717f390de12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:25.922656 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3fa2a-6154-401c-a338-c717f390de12 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:25.938888 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1369dc1-927c-423d-9001-1ebb21e4c4fe None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:26.015504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d013fb1-65b8-454f-a3dc-7c8f71c89efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:26.016557 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d013fb1-65b8-454f-a3dc-7c8f71c89efd None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:26.017049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d013fb1-65b8-454f-a3dc-7c8f71c89efd None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:26.017528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d013fb1-65b8-454f-a3dc-7c8f71c89efd None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:26.018468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d013fb1-65b8-454f-a3dc-7c8f71c89efd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:26.019044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d013fb1-65b8-454f-a3dc-7c8f71c89efd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:26.020737 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d013fb1-65b8-454f-a3dc-7c8f71c89efd None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:26.046064 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 363/709] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:26.052165 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d013fb1-65b8-454f-a3dc-7c8f71c89efd None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:26.088938 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 347/710] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:32:26.094144 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:26.096056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3fa2a-6154-401c-a338-c717f390de12 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:26.206689 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:26.302970 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d013fb1-65b8-454f-a3dc-7c8f71c89efd None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:26.386500 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1df1d9-edd5-4291-808c-5fe9f31001c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:26.387555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1df1d9-edd5-4291-808c-5fe9f31001c2 None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:26.387966 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3fa2a-6154-401c-a338-c717f390de12 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:26.388101 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1df1d9-edd5-4291-808c-5fe9f31001c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:26.388435 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1df1d9-edd5-4291-808c-5fe9f31001c2 None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:26.389556 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1df1d9-edd5-4291-808c-5fe9f31001c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:26.392576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1df1d9-edd5-4291-808c-5fe9f31001c2 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:26.461021 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8310023-6232-46a4-bbc5-69a3a74808a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:26.462462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8310023-6232-46a4-bbc5-69a3a74808a1 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:26.462915 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8310023-6232-46a4-bbc5-69a3a74808a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:26.463356 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8310023-6232-46a4-bbc5-69a3a74808a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:26.476050 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8310023-6232-46a4-bbc5-69a3a74808a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:26.479495 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8310023-6232-46a4-bbc5-69a3a74808a1 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:26.506208 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 364/711] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Aug 8 16:32:26 2022] POST /identity/v3/users => generated 254 bytes in 454 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:32:26.515284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-b8310023-6232-46a4-bbc5-69a3a74808a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:26.515284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:26.515284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:26.515284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:26.515284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:26.515284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:26.515284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:26.515284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:26.515284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:26.515284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:26.515284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:26.515284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:26.515284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:26.515284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:26.515284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:26.516330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:26.516330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:26.516330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:26.516330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:26.516330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:26.516330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:26.516330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:26.516330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:26.516330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:26.516330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:26.516330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:26.516330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:26.516330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:26.516330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:26.516330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:26.516330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:26.516330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:26.517241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:26.517241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:26.517241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:26.517241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:26.517241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:26.517241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:26.517241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:26.517241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:26.518147 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 348/712] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:32:26.525095 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1df1d9-edd5-4291-808c-5fe9f31001c2 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:26.584292 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354a868-607a-46a3-ad5c-571aaa913bda None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:26.584685 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354a868-607a-46a3-ad5c-571aaa913bda None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:26.584941 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354a868-607a-46a3-ad5c-571aaa913bda None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:26.587289 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 349/713] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:26.593672 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5e2ed-246c-4c15-873b-d397dd835fa2 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:26.594007 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5e2ed-246c-4c15-873b-d397dd835fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:26.594276 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5e2ed-246c-4c15-873b-d397dd835fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:26.622669 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-1ff5e2ed-246c-4c15-873b-d397dd835fa2 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:26.624137 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-1ff5e2ed-246c-4c15-873b-d397dd835fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:26.713718 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 350/714] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:32:26.720339 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:26.731717 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1df1d9-edd5-4291-808c-5fe9f31001c2 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:26.740881 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5e2ed-246c-4c15-873b-d397dd835fa2 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:26.782253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6160d81-1cfc-4638-8e18-1c238e429f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:26.782944 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6160d81-1cfc-4638-8e18-1c238e429f20 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:26.783221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6160d81-1cfc-4638-8e18-1c238e429f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:26.783499 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6160d81-1cfc-4638-8e18-1c238e429f20 None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:26.783781 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:26.784239 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6160d81-1cfc-4638-8e18-1c238e429f20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:26.785095 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6160d81-1cfc-4638-8e18-1c238e429f20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:26.787342 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6160d81-1cfc-4638-8e18-1c238e429f20 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:26.809683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 365/715] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:26.815795 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6160d81-1cfc-4638-8e18-1c238e429f20 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:26.880302 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5e2ed-246c-4c15-873b-d397dd835fa2 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:26.934552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6160d81-1cfc-4638-8e18-1c238e429f20 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:26.939298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ae5e5-0db6-4e22-a283-768b7150846f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:26.939641 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ae5e5-0db6-4e22-a283-768b7150846f None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:26.939917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ae5e5-0db6-4e22-a283-768b7150846f None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:26.940259 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ae5e5-0db6-4e22-a283-768b7150846f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:26.940852 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ae5e5-0db6-4e22-a283-768b7150846f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:26.942319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ae5e5-0db6-4e22-a283-768b7150846f None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:26.959133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-003ae5e5-0db6-4e22-a283-768b7150846f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 16:32:26.959133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:26.959133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:32:26.959133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:32:26.959133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:32:26.959133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:32:26.959133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:32:26.959133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:26.959133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 16:32:26.959133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:26.959133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:32:26.959133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:26.959133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:26.959133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:26.959133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:26.959133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:26.959133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:26.960238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:26.960238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:26.960238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:26.960238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:26.960238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:26.960238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:26.960238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:32:26.960238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:32:26.960238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:32:26.960238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:32:26.960238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:26.960238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:26.960238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:26.960238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:26.960238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:26.960238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:26.961092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:26.961092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:26.961092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:26.961092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:26.961092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:26.961092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:26.961092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:26.961092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:26.961092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:26.961092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:26.961092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:32:26.961092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:26.961092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 16:32:26.961092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:26.963576 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 351/716] 10.222.0.13 () {56 vars in 1129 bytes} [Mon Aug 8 16:32:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:26.969309 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ae5e5-0db6-4e22-a283-768b7150846f None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:26.982677 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0436f6-9474-4628-99c3-2277463ce702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:26.983415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0436f6-9474-4628-99c3-2277463ce702 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:26.983692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0436f6-9474-4628-99c3-2277463ce702 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:26.983982 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0436f6-9474-4628-99c3-2277463ce702 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:26.984740 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0436f6-9474-4628-99c3-2277463ce702 None admin] RBAC: Adding query filter params (role.id=e578ef831127463d9b48cc89fe20329c, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=9daa7489bced467984d39e8b8ac49d13) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:26.985052 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0436f6-9474-4628-99c3-2277463ce702 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:26.986583 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0436f6-9474-4628-99c3-2277463ce702 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:26.993348 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 366/717] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:32:26 2022] GET /identity/v3/role_assignments?user.id=9daa7489bced467984d39e8b8ac49d13&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=e578ef831127463d9b48cc89fe20329c => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:27.117649 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:27.125661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ae5e5-0db6-4e22-a283-768b7150846f None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:27.181605 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c3ee3-87dd-4dd4-8d22-ba4981183279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:27.182281 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c3ee3-87dd-4dd4-8d22-ba4981183279 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:27.182584 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c3ee3-87dd-4dd4-8d22-ba4981183279 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:27.182873 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c3ee3-87dd-4dd4-8d22-ba4981183279 None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:27.183493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c3ee3-87dd-4dd4-8d22-ba4981183279 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:27.183789 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c3ee3-87dd-4dd4-8d22-ba4981183279 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:27.185839 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c3ee3-87dd-4dd4-8d22-ba4981183279 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:27.195016 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 352/718] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Aug 8 16:32:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:32:27.204120 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0436f6-9474-4628-99c3-2277463ce702 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:27.223015 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:27.321215 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0436f6-9474-4628-99c3-2277463ce702 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:27.360827 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955bc06-1d22-4a8f-8c92-d4271422fa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:27.361852 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955bc06-1d22-4a8f-8c92-d4271422fa87 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:27.362336 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955bc06-1d22-4a8f-8c92-d4271422fa87 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:27.362883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955bc06-1d22-4a8f-8c92-d4271422fa87 None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:27.364040 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d955bc06-1d22-4a8f-8c92-d4271422fa87 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:27.364589 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d955bc06-1d22-4a8f-8c92-d4271422fa87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:27.367205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d955bc06-1d22-4a8f-8c92-d4271422fa87 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:27.373892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 367/719] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Aug 8 16:32:27 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:27.500992 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec242d-72e5-4f0a-861f-9dac3f913043 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:27.501233 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec242d-72e5-4f0a-861f-9dac3f913043 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:27.501499 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec242d-72e5-4f0a-861f-9dac3f913043 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:27.503292 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 353/720] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:27.508959 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddeb84-caf0-4ed6-bd97-d175b1b5f1ad None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:27.509295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddeb84-caf0-4ed6-bd97-d175b1b5f1ad None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:27.509647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddeb84-caf0-4ed6-bd97-d175b1b5f1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:27.535750 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-12ddeb84-caf0-4ed6-bd97-d175b1b5f1ad None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:27.536985 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-12ddeb84-caf0-4ed6-bd97-d175b1b5f1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:27.639200 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 368/721] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:32:27.667253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec242d-72e5-4f0a-861f-9dac3f913043 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:27.807485 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec242d-72e5-4f0a-861f-9dac3f913043 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:27.864577 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c9a5d-51d7-479e-8b8b-0740fafc2de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:27.865194 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c9a5d-51d7-479e-8b8b-0740fafc2de3 None admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:27.865517 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c9a5d-51d7-479e-8b8b-0740fafc2de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:27.865789 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c9a5d-51d7-479e-8b8b-0740fafc2de3 None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:27.866437 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c9a5d-51d7-479e-8b8b-0740fafc2de3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:27.867954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c9a5d-51d7-479e-8b8b-0740fafc2de3 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:27.988517 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 354/722] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Aug 8 16:32:27 2022] POST /identity/v3/services => generated 240 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:32:28.092964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68c459-2c80-4caf-9742-45e33ea12307 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:28.093943 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68c459-2c80-4caf-9742-45e33ea12307 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:28.094423 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68c459-2c80-4caf-9742-45e33ea12307 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:28.096457 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 369/723] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:28.102681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48903489-3828-47c0-8057-65e5eb5fc99b None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:28.103398 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48903489-3828-47c0-8057-65e5eb5fc99b None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:28.103878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48903489-3828-47c0-8057-65e5eb5fc99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:28.132714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-48903489-3828-47c0-8057-65e5eb5fc99b None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:28.135011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-48903489-3828-47c0-8057-65e5eb5fc99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:28.158904 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:28.235861 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 355/724] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:28 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:32:28.259611 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68c459-2c80-4caf-9742-45e33ea12307 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:28.271999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:28.384576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68c459-2c80-4caf-9742-45e33ea12307 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:28.425535 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f8b168-9341-43a7-ac9e-6e2e6c6b76af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:28.426244 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8b168-9341-43a7-ac9e-6e2e6c6b76af None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:28.426526 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8b168-9341-43a7-ac9e-6e2e6c6b76af None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:28.426831 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8b168-9341-43a7-ac9e-6e2e6c6b76af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:28.439962 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f8b168-9341-43a7-ac9e-6e2e6c6b76af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:32:28.444862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-d4f8b168-9341-43a7-ac9e-6e2e6c6b76af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:32:28.444862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:28.444862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:28.444862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:28.444862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:28.444862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:28.444862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:28.444862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:28.444862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:28.444862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:28.444862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:28.444862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:28.444862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:32:28.444862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:32:28.444862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:32:28.444862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:32:28.445881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:32:28.445881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:32:28.445881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:32:28.445881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:32:28.445881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:28.445881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:28.445881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:28.445881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:28.445881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:28.445881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:28.445881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:28.445881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:28.445881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:28.445881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:28.445881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:28.445881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:28.448085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:28.448085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:28.448085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:28.448085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:28.448085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:32:28.448085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:32:28.448085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:32:28.448085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:32:28.448085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:32:28.448085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:32:28.448085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:32:28.448085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:28.448085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 370/725] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:32:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:28.453711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48903489-3828-47c0-8057-65e5eb5fc99b None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:28.570860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48903489-3828-47c0-8057-65e5eb5fc99b None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:28.609348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df04e4-cb43-4f63-810f-64e2d04828f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:28.610025 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df04e4-cb43-4f63-810f-64e2d04828f1 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:28.610303 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df04e4-cb43-4f63-810f-64e2d04828f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:28.610588 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df04e4-cb43-4f63-810f-64e2d04828f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:28.611700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df04e4-cb43-4f63-810f-64e2d04828f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:28.612234 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df04e4-cb43-4f63-810f-64e2d04828f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:28.615143 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df04e4-cb43-4f63-810f-64e2d04828f1 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:28.633464 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 356/726] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:32:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:28.643136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f8b168-9341-43a7-ac9e-6e2e6c6b76af None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:28.751917 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:28.801952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f8b168-9341-43a7-ac9e-6e2e6c6b76af None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:28.850039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32b780-42fc-494a-8711-1a471fba3375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:28.850713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32b780-42fc-494a-8711-1a471fba3375 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:28.850995 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32b780-42fc-494a-8711-1a471fba3375 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:28.851268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32b780-42fc-494a-8711-1a471fba3375 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:28.865702 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e32b780-42fc-494a-8711-1a471fba3375 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:28.867209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e32b780-42fc-494a-8711-1a471fba3375 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:28.887226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-1e32b780-42fc-494a-8711-1a471fba3375 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:28.887226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:28.887226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:28.887226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:28.887226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:28.887226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:28.887226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:28.887226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:28.887226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:28.887226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:28.887226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:28.887226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:28.887226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:28.887226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:28.887226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:28.887226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:28.888307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:28.888307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:28.888307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:28.888307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:28.888307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:28.888307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:28.888307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:28.888307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:28.888307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:28.888307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:28.888307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:28.888307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:28.888307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:28.888307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:28.888307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:28.888307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:28.888307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:28.889267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:28.889267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:28.889267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:28.889267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 371/727] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:28.894525 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df04e4-cb43-4f63-810f-64e2d04828f1 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:28.905320 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:29.063140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df04e4-cb43-4f63-810f-64e2d04828f1 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:29.109793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f1cde-9dbf-49fe-aa33-bc79124cd1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:29.110862 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f1cde-9dbf-49fe-aa33-bc79124cd1b0 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:29.111348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f1cde-9dbf-49fe-aa33-bc79124cd1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:29.111883 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f1cde-9dbf-49fe-aa33-bc79124cd1b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:29.112775 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f1cde-9dbf-49fe-aa33-bc79124cd1b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:29.113293 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f1cde-9dbf-49fe-aa33-bc79124cd1b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:29.114949 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f1cde-9dbf-49fe-aa33-bc79124cd1b0 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:29.138129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 357/728] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:29.143820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32b780-42fc-494a-8711-1a471fba3375 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:29.305848 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32b780-42fc-494a-8711-1a471fba3375 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:29.337255 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:29.354183 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f838f-c601-43a3-bcab-6182fc38971e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:29.354893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f838f-c601-43a3-bcab-6182fc38971e None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:29.355175 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f838f-c601-43a3-bcab-6182fc38971e None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:29.355448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f838f-c601-43a3-bcab-6182fc38971e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:29.364505 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f838f-c601-43a3-bcab-6182fc38971e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:29.366057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f838f-c601-43a3-bcab-6182fc38971e None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:29.379561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-2f0f838f-c601-43a3-bcab-6182fc38971e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:32:29.379561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:29.379561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:29.379561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:29.379561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:29.379561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:29.379561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:29.379561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:29.379561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:29.379561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:29.379561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:29.379561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:29.379561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:29.379561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:29.379561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:29.379561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:29.380637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:29.380637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:29.380637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:29.380637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:29.380637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:29.380637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:29.380637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:29.380637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:29.380637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:29.380637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:29.380637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:29.380637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:29.380637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:29.380637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:29.380637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:29.380637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:29.380637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:29.380637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:29.381614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:29.381614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:29.381614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:29.381614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:29.381614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:29.381614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:29.381614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:29.381614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:29.381614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:32:29.381614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:29.381614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 372/729] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:32:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:29.387387 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f1cde-9dbf-49fe-aa33-bc79124cd1b0 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:29.540485 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:29.621047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f1cde-9dbf-49fe-aa33-bc79124cd1b0 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:29.684480 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ec3da-6365-4e98-a90a-9d041c4db7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:29.685490 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ec3da-6365-4e98-a90a-9d041c4db7fa None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:29.685965 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ec3da-6365-4e98-a90a-9d041c4db7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:29.686441 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ec3da-6365-4e98-a90a-9d041c4db7fa None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:29.687823 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ec3da-6365-4e98-a90a-9d041c4db7fa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:29.688352 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ec3da-6365-4e98-a90a-9d041c4db7fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:29.691213 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ec3da-6365-4e98-a90a-9d041c4db7fa None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:29.726181 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 358/730] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Aug 8 16:32:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:29.734929 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f838f-c601-43a3-bcab-6182fc38971e None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:29.869602 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f5f8e-2306-46a1-b73a-4652eb506f45 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:29.870289 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f5f8e-2306-46a1-b73a-4652eb506f45 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:29.870849 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f5f8e-2306-46a1-b73a-4652eb506f45 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:29.872741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 359/731] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:32:29.878550 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a9d2a-922e-4703-9664-f9e4aca5ece4 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:29.879238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a9d2a-922e-4703-9664-f9e4aca5ece4 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:29.879720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a9d2a-922e-4703-9664-f9e4aca5ece4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:29.900692 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-409a9d2a-922e-4703-9664-f9e4aca5ece4 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:29.902811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-409a9d2a-922e-4703-9664-f9e4aca5ece4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:29.965688 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f838f-c601-43a3-bcab-6182fc38971e None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:30.023256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36cb5f-6f65-41cb-bd25-d343614979b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:30.023902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36cb5f-6f65-41cb-bd25-d343614979b1 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:30.025726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36cb5f-6f65-41cb-bd25-d343614979b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:30.025726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36cb5f-6f65-41cb-bd25-d343614979b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:30.031941 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 360/732] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:29 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:32:30.033524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed772ad5-97fb-4fd5-b9b6-117923cdf278 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:30.034144 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed772ad5-97fb-4fd5-b9b6-117923cdf278 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:30.034662 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed772ad5-97fb-4fd5-b9b6-117923cdf278 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:30.036556 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 361/733] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:30.044126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61855625-bbe9-41a0-8c85-a66abf9e21e2 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:30.044516 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61855625-bbe9-41a0-8c85-a66abf9e21e2 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:30.044691 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61855625-bbe9-41a0-8c85-a66abf9e21e2 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:30.046458 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 362/734] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:30.049760 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e1d37-4bc5-445b-b663-ef7bc7f02869 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:30.050057 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e1d37-4bc5-445b-b663-ef7bc7f02869 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:30.050313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e1d37-4bc5-445b-b663-ef7bc7f02869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:30.070001 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-af6e1d37-4bc5-445b-b663-ef7bc7f02869 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:30.071794 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-af6e1d37-4bc5-445b-b663-ef7bc7f02869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:30.107656 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:30.142619 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36cb5f-6f65-41cb-bd25-d343614979b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:30.142619 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36cb5f-6f65-41cb-bd25-d343614979b1 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:30.143011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:30.159876 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 363/735] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:30 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:32:30.164277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460039a9-54da-45ce-b510-6edca381a463 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:30.165522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460039a9-54da-45ce-b510-6edca381a463 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:30.166235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460039a9-54da-45ce-b510-6edca381a463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:30.200157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-460039a9-54da-45ce-b510-6edca381a463 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:30.202059 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-460039a9-54da-45ce-b510-6edca381a463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:30.268736 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 373/736] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:32:29 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:32:30.268736 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36cb5f-6f65-41cb-bd25-d343614979b1 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:30.314816 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 364/737] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:30 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:32:30.314816 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460039a9-54da-45ce-b510-6edca381a463 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:30.422985 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:30.453156 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36cb5f-6f65-41cb-bd25-d343614979b1 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:30.486454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460039a9-54da-45ce-b510-6edca381a463 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:30.499737 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2424f-3b61-4bbb-83c1-e57d75981505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:30.500563 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2424f-3b61-4bbb-83c1-e57d75981505 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:30.500967 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2424f-3b61-4bbb-83c1-e57d75981505 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:30.501380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2424f-3b61-4bbb-83c1-e57d75981505 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:30.516151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2424f-3b61-4bbb-83c1-e57d75981505 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:32:30.518047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:30.522808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-58a2424f-3b61-4bbb-83c1-e57d75981505 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:30.522808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:30.522808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:30.522808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:30.522808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:30.522808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:30.522808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:30.522808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:30.522808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:30.522808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:30.522808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:30.522808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:30.522808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:32:30.522808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:32:30.522808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:32:30.522808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:32:30.524915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:32:30.524915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:32:30.524915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:32:30.524915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:32:30.524915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:30.524915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:30.524915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:30.524915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:30.524915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:30.524915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:30.524915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:30.524915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:30.524915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:30.524915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:30.524915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:30.524915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:30.527978 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:30.527978 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:30.527978 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:30.527978 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:30.527978 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:32:30.527978 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:32:30.527978 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:32:30.527978 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:32:30.527978 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:32:30.527978 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:32:30.527978 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:30.527978 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:30.527978 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 374/738] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:32:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:30.527978 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2424f-3b61-4bbb-83c1-e57d75981505 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:30.567515 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6272db-b167-4ac8-8102-d17c72bb3f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:30.568569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6272db-b167-4ac8-8102-d17c72bb3f62 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:30.569034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6272db-b167-4ac8-8102-d17c72bb3f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:30.569484 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6272db-b167-4ac8-8102-d17c72bb3f62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:30.590412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6272db-b167-4ac8-8102-d17c72bb3f62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:32:30.595425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-4e6272db-b167-4ac8-8102-d17c72bb3f62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:32:30.595425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:30.595425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:30.595425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:30.595425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:30.595425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:30.595425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:30.595425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:30.595425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:30.595425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:30.595425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:30.595425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:30.595425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:32:30.595425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:32:30.595425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:32:30.595425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:32:30.597358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:32:30.597358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:32:30.597358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:32:30.597358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:32:30.597358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:30.597358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:30.597358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:30.597358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:30.597358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:30.597358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:30.597358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:30.597358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:30.597358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:30.597358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:30.597358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:30.597358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:30.599109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:30.599109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:30.599109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:30.599109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:30.599109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:32:30.599109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:32:30.599109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:32:30.599109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:32:30.599109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:32:30.599109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:32:30.599109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:32:30.599109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:30.601885 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 365/739] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:32:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:30.602748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6272db-b167-4ac8-8102-d17c72bb3f62 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:30.665764 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2424f-3b61-4bbb-83c1-e57d75981505 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:30.721013 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8018053-e113-42c4-a6d1-b01285754d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:30.721937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8018053-e113-42c4-a6d1-b01285754d9f None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:30.722379 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8018053-e113-42c4-a6d1-b01285754d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:30.722862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8018053-e113-42c4-a6d1-b01285754d9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:30.736693 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:30.739910 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8018053-e113-42c4-a6d1-b01285754d9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:30.742470 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8018053-e113-42c4-a6d1-b01285754d9f None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:30.756550 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6272db-b167-4ac8-8102-d17c72bb3f62 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:30.769280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-c8018053-e113-42c4-a6d1-b01285754d9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:30.769280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:30.769280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:30.769280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:30.769280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:30.769280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:30.769280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:30.769280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:30.769280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:30.769280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:30.769280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:30.769280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:30.769280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:30.769280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:30.769280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:30.769280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:30.771490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:30.771490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:30.771490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:30.771490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:30.771490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:30.771490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:30.771490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:30.771490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:30.771490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:30.771490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:30.771490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:30.771490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:30.771490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:30.771490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:30.771490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:30.771490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:30.771490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:30.773258 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:30.773258 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:30.773258 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:30.777764 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 375/740] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:30.778920 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8018053-e113-42c4-a6d1-b01285754d9f None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:30.810317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b59302-d2ba-494a-9e11-10f69d3ecc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:30.811718 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b59302-d2ba-494a-9e11-10f69d3ecc9c None admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:30.813348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b59302-d2ba-494a-9e11-10f69d3ecc9c None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:30.813348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b59302-d2ba-494a-9e11-10f69d3ecc9c None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:30.813971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b59302-d2ba-494a-9e11-10f69d3ecc9c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:30.816109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b59302-d2ba-494a-9e11-10f69d3ecc9c None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:30.889396 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 366/741] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Aug 8 16:32:30 2022] POST /identity/v3/services => generated 251 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:32:30.890255 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b59302-d2ba-494a-9e11-10f69d3ecc9c None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:30.912091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8018053-e113-42c4-a6d1-b01285754d9f None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:30.946139 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:30.992286 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f4836-4eea-4bfa-894c-75cd3c6110fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:30.994022 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f4836-4eea-4bfa-894c-75cd3c6110fd None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:30.994539 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f4836-4eea-4bfa-894c-75cd3c6110fd None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:30.995020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f4836-4eea-4bfa-894c-75cd3c6110fd None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:30.996180 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f4836-4eea-4bfa-894c-75cd3c6110fd None admin] RBAC: Adding query filter params (name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:30.996691 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f4836-4eea-4bfa-894c-75cd3c6110fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:30.998896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f4836-4eea-4bfa-894c-75cd3c6110fd None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:31.013545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 376/742] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Aug 8 16:32:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:31.015030 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f4836-4eea-4bfa-894c-75cd3c6110fd None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:31.080634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b59302-d2ba-494a-9e11-10f69d3ecc9c None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:31.101817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:31.138059 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a80a4-e019-4f7e-aee9-e69f0af26671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:31.139113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a80a4-e019-4f7e-aee9-e69f0af26671 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:31.139577 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a80a4-e019-4f7e-aee9-e69f0af26671 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:31.140046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a80a4-e019-4f7e-aee9-e69f0af26671 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:31.141245 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a80a4-e019-4f7e-aee9-e69f0af26671 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:31.141765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a80a4-e019-4f7e-aee9-e69f0af26671 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:31.144328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a80a4-e019-4f7e-aee9-e69f0af26671 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:31.163725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 367/743] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:32:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:31.163725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db720b-fe7f-4710-927a-e4bd4b03b9ef None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:31.163725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db720b-fe7f-4710-927a-e4bd4b03b9ef None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:31.163725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db720b-fe7f-4710-927a-e4bd4b03b9ef None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:31.165938 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 368/744] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:31.169851 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db720b-fe7f-4710-927a-e4bd4b03b9ef None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:31.177088 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f4836-4eea-4bfa-894c-75cd3c6110fd None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:31.218872 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61766217-e8f2-4cf7-8c3d-aa76ebb1fced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:31.219920 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61766217-e8f2-4cf7-8c3d-aa76ebb1fced None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:31.220202 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61766217-e8f2-4cf7-8c3d-aa76ebb1fced None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:31.220512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61766217-e8f2-4cf7-8c3d-aa76ebb1fced None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:31.221112 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61766217-e8f2-4cf7-8c3d-aa76ebb1fced None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:31.221410 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61766217-e8f2-4cf7-8c3d-aa76ebb1fced None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:31.222823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61766217-e8f2-4cf7-8c3d-aa76ebb1fced None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:31.238053 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 377/745] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:31.239355 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61766217-e8f2-4cf7-8c3d-aa76ebb1fced None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:31.299511 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:31.307863 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db720b-fe7f-4710-927a-e4bd4b03b9ef None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:31.345337 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8bf00-615a-44f7-a5e9-aac5d0769082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:31.345977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8bf00-615a-44f7-a5e9-aac5d0769082 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:31.346252 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8bf00-615a-44f7-a5e9-aac5d0769082 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:31.346598 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8bf00-615a-44f7-a5e9-aac5d0769082 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:31.358340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8bf00-615a-44f7-a5e9-aac5d0769082 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:31.359849 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8bf00-615a-44f7-a5e9-aac5d0769082 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:31.376874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-75a8bf00-615a-44f7-a5e9-aac5d0769082 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:31.376874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:31.376874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:31.376874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:31.376874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:31.376874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:31.376874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:31.376874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:31.376874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:31.376874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:31.376874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:31.376874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:31.376874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:31.376874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:31.376874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:31.376874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:31.377900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:31.377900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:31.377900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:31.377900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:31.377900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:31.377900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:31.377900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:31.377900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:31.377900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:31.377900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:31.377900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:31.377900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:31.377900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:31.377900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:31.377900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:31.377900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:31.377900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:31.383738 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:31.383738 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:31.383738 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:31.383738 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 369/746] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:31.383738 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c9a2e-c628-43b5-82aa-71f61c3a4bb3 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:31.383738 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c9a2e-c628-43b5-82aa-71f61c3a4bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:31.383738 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c9a2e-c628-43b5-82aa-71f61c3a4bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:31.402248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-647c9a2e-c628-43b5-82aa-71f61c3a4bb3 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:31.403495 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-647c9a2e-c628-43b5-82aa-71f61c3a4bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:31.435869 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61766217-e8f2-4cf7-8c3d-aa76ebb1fced None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:31.450682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:31.487259 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725bf6b4-0db2-4d59-b5ef-4a9c2040276e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:31.488538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bf6b4-0db2-4d59-b5ef-4a9c2040276e None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:31.489401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bf6b4-0db2-4d59-b5ef-4a9c2040276e None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:31.489516 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bf6b4-0db2-4d59-b5ef-4a9c2040276e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:31.512386 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725bf6b4-0db2-4d59-b5ef-4a9c2040276e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:31.515169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725bf6b4-0db2-4d59-b5ef-4a9c2040276e None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:31.530655 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 370/747] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:32:31.530655 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c9a2e-c628-43b5-82aa-71f61c3a4bb3 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:31.549380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-725bf6b4-0db2-4d59-b5ef-4a9c2040276e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:31.549380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:31.549380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:31.549380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:31.549380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:31.549380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:31.549380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:31.549380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:31.549380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:31.549380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:31.549380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:31.549380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:31.549380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:31.549380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:31.549380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:31.549380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:31.550783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:31.550783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:31.550783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:31.550783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:31.550783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:31.550783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:31.550783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:31.550783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:31.550783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:31.550783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:31.550783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:31.550783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:31.550783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:31.550783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:31.550783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:31.550783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:31.550783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:31.551959 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:31.551959 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:31.551959 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:31.553875 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 378/748] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:31.556209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725bf6b4-0db2-4d59-b5ef-4a9c2040276e None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:31.664824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:31.698522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c9a2e-c628-43b5-82aa-71f61c3a4bb3 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:31.746215 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725bf6b4-0db2-4d59-b5ef-4a9c2040276e None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:31.759642 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e27a7-54f6-402d-b636-20a4aafb88fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:31.760735 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e27a7-54f6-402d-b636-20a4aafb88fe None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:31.761379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e27a7-54f6-402d-b636-20a4aafb88fe None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:31.762104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e27a7-54f6-402d-b636-20a4aafb88fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:31.776840 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e27a7-54f6-402d-b636-20a4aafb88fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:31.780078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e27a7-54f6-402d-b636-20a4aafb88fe None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:31.792604 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5f90c-bb5b-4148-ad66-94ab946734c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:31.793331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5f90c-bb5b-4148-ad66-94ab946734c7 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:31.793745 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5f90c-bb5b-4148-ad66-94ab946734c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:31.794078 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5f90c-bb5b-4148-ad66-94ab946734c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:31.794909 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5f90c-bb5b-4148-ad66-94ab946734c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:31.795269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5f90c-bb5b-4148-ad66-94ab946734c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:31.796731 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5f90c-bb5b-4148-ad66-94ab946734c7 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:31.806218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-7b4e27a7-54f6-402d-b636-20a4aafb88fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:31.806218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:31.806218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:31.806218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:31.806218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:31.806218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:31.806218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:31.806218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:31.806218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:31.806218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:31.806218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:31.806218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:31.806218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:31.806218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:31.806218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:31.808027 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:31.808027 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:31.808027 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:31.808027 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:31.808027 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:31.808027 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:31.808027 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:31.808027 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:31.808027 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:31.808027 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:31.808027 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:31.808027 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:31.808027 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:31.808027 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:31.808027 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:31.808027 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:31.808027 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:31.809806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:31.809806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:31.809806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:31.809806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:31.809806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:31.809806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:31.809806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:31.809806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:31.809806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 371/749] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:31.810796 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e27a7-54f6-402d-b636-20a4aafb88fe None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:31.817497 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 379/750] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:31.819677 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5f90c-bb5b-4148-ad66-94ab946734c7 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:31.856494 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:31.985509 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5f90c-bb5b-4148-ad66-94ab946734c7 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:31.999305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e27a7-54f6-402d-b636-20a4aafb88fe None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:32.037816 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403fafe-2e43-4fe0-b248-6b5681a0a040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:32.038107 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403fafe-2e43-4fe0-b248-6b5681a0a040 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:32.038291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403fafe-2e43-4fe0-b248-6b5681a0a040 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:32.038686 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403fafe-2e43-4fe0-b248-6b5681a0a040 None admin] PATH_INFO: `/v3/services/network` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:32.039160 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5403fafe-2e43-4fe0-b248-6b5681a0a040 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:32.041363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5403fafe-2e43-4fe0-b248-6b5681a0a040 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:32.051401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-5403fafe-2e43-4fe0-b248-6b5681a0a040 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 16:32:32.051401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:32.051401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:32:32.051401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:32:32.051401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:32:32.051401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:32:32.051401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:32:32.051401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:32.051401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 16:32:32.051401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:32.051401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:32:32.051401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:32.051401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:32.051401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:32.051401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:32.051401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:32.051401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:32.053382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:32.053382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:32.053382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:32.053382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:32.053382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:32.053382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:32.053382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:32:32.053382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:32:32.053382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:32:32.053382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:32:32.053382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:32.053382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:32.053382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:32.053382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:32.053382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:32.053382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:32.055087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:32.055087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:32.055087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:32.055087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:32.055087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:32.055087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:32.055087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:32.055087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:32.055087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:32.055087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:32.055087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:32:32.055087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:32.055087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 16:32:32.055087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:32.055087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 372/751] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:31 2022] GET /identity/v3/services/network => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:32.057269 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403fafe-2e43-4fe0-b248-6b5681a0a040 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:32.062371 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:32.067623 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c1c63-7207-44a5-ba26-c27917b44bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:32.068701 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c1c63-7207-44a5-ba26-c27917b44bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:32.069865 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c1c63-7207-44a5-ba26-c27917b44bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:32.070349 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c1c63-7207-44a5-ba26-c27917b44bf3 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:32.072392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648c1c63-7207-44a5-ba26-c27917b44bf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:32.072931 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648c1c63-7207-44a5-ba26-c27917b44bf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:32.075602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648c1c63-7207-44a5-ba26-c27917b44bf3 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:32.102192 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 380/752] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:32.103641 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c1c63-7207-44a5-ba26-c27917b44bf3 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:32.235668 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:32.237601 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403fafe-2e43-4fe0-b248-6b5681a0a040 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:32.272478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c1c63-7207-44a5-ba26-c27917b44bf3 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:32.289280 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c3edc-39eb-4f59-bbfb-72bb860510a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:32.290362 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c3edc-39eb-4f59-bbfb-72bb860510a2 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:32.290681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c3edc-39eb-4f59-bbfb-72bb860510a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:32.291007 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c3edc-39eb-4f59-bbfb-72bb860510a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:32.291653 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c3edc-39eb-4f59-bbfb-72bb860510a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:32.291951 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c3edc-39eb-4f59-bbfb-72bb860510a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:32.293852 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c3edc-39eb-4f59-bbfb-72bb860510a2 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:32.316594 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 373/753] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:32.316594 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c3edc-39eb-4f59-bbfb-72bb860510a2 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:32.331206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7ec23-e264-4389-862a-f114859d9e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:32.331936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7ec23-e264-4389-862a-f114859d9e4b None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:32.332263 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7ec23-e264-4389-862a-f114859d9e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:32.332542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7ec23-e264-4389-862a-f114859d9e4b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:32.342739 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd7ec23-e264-4389-862a-f114859d9e4b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:32.344197 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd7ec23-e264-4389-862a-f114859d9e4b None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:32.357545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-8bd7ec23-e264-4389-862a-f114859d9e4b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:32:32.357545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:32.357545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:32.357545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:32.357545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:32.357545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:32.357545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:32.357545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:32.357545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:32.357545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:32.357545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:32.357545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:32.357545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:32.357545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:32.357545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:32.357545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:32.358636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:32.358636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:32.358636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:32.358636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:32.358636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:32.358636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:32.358636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:32.358636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:32.358636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:32.358636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:32.358636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:32.358636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:32.358636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:32.358636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:32.358636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:32.358636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:32.358636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:32.358636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:32.361172 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:32.361172 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:32.361172 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:32.361172 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:32.361172 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:32.361172 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:32.361172 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:32.361172 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:32.361172 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:32:32.361172 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:32.363574 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 381/754] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Aug 8 16:32:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:32.364521 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7ec23-e264-4389-862a-f114859d9e4b None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:32.404491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:32.471305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c3edc-39eb-4f59-bbfb-72bb860510a2 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:32.511602 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73ada2-7332-4f7d-abde-ba4050cc0558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:32.512216 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73ada2-7332-4f7d-abde-ba4050cc0558 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:32.512493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73ada2-7332-4f7d-abde-ba4050cc0558 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:32.512897 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73ada2-7332-4f7d-abde-ba4050cc0558 None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:32.513444 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73ada2-7332-4f7d-abde-ba4050cc0558 None admin] RBAC: Adding query filter params (name=network) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:32.513743 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73ada2-7332-4f7d-abde-ba4050cc0558 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:32.515209 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73ada2-7332-4f7d-abde-ba4050cc0558 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:32.520347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 374/755] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Aug 8 16:32:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:32.521197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73ada2-7332-4f7d-abde-ba4050cc0558 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:32.576327 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7ec23-e264-4389-862a-f114859d9e4b None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:32.632207 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b912cb-ac22-437b-bc51-1b9803bcbef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:32.633277 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b912cb-ac22-437b-bc51-1b9803bcbef5 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:32.634383 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b912cb-ac22-437b-bc51-1b9803bcbef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:32.634867 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b912cb-ac22-437b-bc51-1b9803bcbef5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:32.647537 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b912cb-ac22-437b-bc51-1b9803bcbef5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:32.650092 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b912cb-ac22-437b-bc51-1b9803bcbef5 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:32.671431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-b3b912cb-ac22-437b-bc51-1b9803bcbef5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:32:32.671431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:32.671431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:32.671431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:32.671431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:32.671431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:32.671431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:32.671431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:32.671431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:32.671431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:32.671431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:32.671431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:32.671431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:32.671431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:32.671431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:32.671431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:32.673292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:32.673292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:32.673292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:32.673292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:32.673292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:32.673292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:32.673292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:32.673292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:32.673292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:32.673292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:32.673292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:32.673292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:32.673292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:32.673292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:32.673292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:32.673292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:32.673292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:32.673292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:32.675986 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:32.675986 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:32.675986 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:32.675986 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:32.675986 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:32.675986 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:32.675986 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:32.675986 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:32.675986 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:32:32.675986 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:32.675986 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 382/756] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Aug 8 16:32:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:32.675986 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b912cb-ac22-437b-bc51-1b9803bcbef5 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:32.679431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:32.695522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73ada2-7332-4f7d-abde-ba4050cc0558 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:32.742149 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:32.756716 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358b93b-2f5c-4d98-afb5-208acef8c280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:32.757680 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358b93b-2f5c-4d98-afb5-208acef8c280 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:32.758126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358b93b-2f5c-4d98-afb5-208acef8c280 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:32.758535 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358b93b-2f5c-4d98-afb5-208acef8c280 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:32.759819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358b93b-2f5c-4d98-afb5-208acef8c280 None admin] RBAC: Adding query filter params (role.id=e578ef831127463d9b48cc89fe20329c, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=c606decb832444e38a194b98f8d15f8c) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:32.760331 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358b93b-2f5c-4d98-afb5-208acef8c280 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:32.762688 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358b93b-2f5c-4d98-afb5-208acef8c280 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:32.772303 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 375/757] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:32:32 2022] GET /identity/v3/role_assignments?user.id=c606decb832444e38a194b98f8d15f8c&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=e578ef831127463d9b48cc89fe20329c => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:32.773637 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358b93b-2f5c-4d98-afb5-208acef8c280 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:32.876599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b912cb-ac22-437b-bc51-1b9803bcbef5 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:32.927906 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548a551-d37b-4c12-b8f8-8e500f582b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:32.932725 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548a551-d37b-4c12-b8f8-8e500f582b0f None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:32.933127 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548a551-d37b-4c12-b8f8-8e500f582b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:32.933401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548a551-d37b-4c12-b8f8-8e500f582b0f None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:32.934233 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6548a551-d37b-4c12-b8f8-8e500f582b0f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:32.934545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6548a551-d37b-4c12-b8f8-8e500f582b0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:32.936090 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6548a551-d37b-4c12-b8f8-8e500f582b0f None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:32.968466 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358b93b-2f5c-4d98-afb5-208acef8c280 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:32.975690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 383/758] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Aug 8 16:32:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:32.976389 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548a551-d37b-4c12-b8f8-8e500f582b0f None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:33.041415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9719bcc-5165-4697-ac5d-71d15127d577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:33.043486 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9719bcc-5165-4697-ac5d-71d15127d577 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:33.044233 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9719bcc-5165-4697-ac5d-71d15127d577 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:33.045020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9719bcc-5165-4697-ac5d-71d15127d577 None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:33.046375 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9719bcc-5165-4697-ac5d-71d15127d577 None admin] RBAC: Adding query filter params (type=network) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:33.047204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9719bcc-5165-4697-ac5d-71d15127d577 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:33.049891 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9719bcc-5165-4697-ac5d-71d15127d577 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:33.069704 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 376/759] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Aug 8 16:32:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:33.072379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9719bcc-5165-4697-ac5d-71d15127d577 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:33.097327 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:33.176352 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:33.181900 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548a551-d37b-4c12-b8f8-8e500f582b0f None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:33.260632 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b45a1-4b5b-4598-ac41-00261aff46b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:33.262434 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b45a1-4b5b-4598-ac41-00261aff46b5 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:33.263192 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b45a1-4b5b-4598-ac41-00261aff46b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:33.264211 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b45a1-4b5b-4598-ac41-00261aff46b5 None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:33.266414 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b45a1-4b5b-4598-ac41-00261aff46b5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:33.267547 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b45a1-4b5b-4598-ac41-00261aff46b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:33.270087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9719bcc-5165-4697-ac5d-71d15127d577 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:33.270730 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b45a1-4b5b-4598-ac41-00261aff46b5 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:33.310172 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 384/760] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Aug 8 16:32:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:33.312250 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b45a1-4b5b-4598-ac41-00261aff46b5 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:33.350601 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baae7e7-5130-4008-af08-48b8a1c85bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:33.351804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baae7e7-5130-4008-af08-48b8a1c85bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:33.352313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baae7e7-5130-4008-af08-48b8a1c85bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:33.352772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baae7e7-5130-4008-af08-48b8a1c85bf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:33.503181 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b45a1-4b5b-4598-ac41-00261aff46b5 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:33.507788 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baae7e7-5130-4008-af08-48b8a1c85bf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:33.511007 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baae7e7-5130-4008-af08-48b8a1c85bf4 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:33.523054 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:33.543523 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0cf71-21bd-4e4c-ad23-2e577c9b1eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:33.544695 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0cf71-21bd-4e4c-ad23-2e577c9b1eec None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:33.545187 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0cf71-21bd-4e4c-ad23-2e577c9b1eec None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:33.545646 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0cf71-21bd-4e4c-ad23-2e577c9b1eec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:33.546988 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e0cf71-21bd-4e4c-ad23-2e577c9b1eec None admin] RBAC: Adding query filter params (interface=public, service_id=36c4c60c060447a5aa9e5c93d8ed21ac, region_id=RegionOne) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:33.547524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e0cf71-21bd-4e4c-ad23-2e577c9b1eec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:33.550304 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e0cf71-21bd-4e4c-ad23-2e577c9b1eec None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:33.571353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 385/761] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Aug 8 16:32:33 2022] GET /identity/v3/endpoints?service_id=36c4c60c060447a5aa9e5c93d8ed21ac&interface=public®ion_id=RegionOne => generated 194 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:33.572969 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0cf71-21bd-4e4c-ad23-2e577c9b1eec None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:33.638404 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:33.734335 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 377/762] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:32:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:32:33.736662 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baae7e7-5130-4008-af08-48b8a1c85bf4 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:33.784560 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0cf71-21bd-4e4c-ad23-2e577c9b1eec None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:33.853058 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a588f-6797-4ea4-915c-93b7420ba61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:33.853712 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a588f-6797-4ea4-915c-93b7420ba61a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:33.853988 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a588f-6797-4ea4-915c-93b7420ba61a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:33.854261 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a588f-6797-4ea4-915c-93b7420ba61a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:33.868120 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a588f-6797-4ea4-915c-93b7420ba61a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:33.869613 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a588f-6797-4ea4-915c-93b7420ba61a None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:33.888290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-b82a588f-6797-4ea4-915c-93b7420ba61a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:33.888290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:33.888290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:33.888290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:33.888290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:33.888290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:33.888290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:33.888290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:33.888290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:33.888290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:33.888290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:33.888290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:33.888290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:33.888290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:33.888290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:33.888290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:33.890160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:33.890160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:33.890160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:33.890160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:33.890160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:33.890160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:33.890160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:33.890160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:33.890160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:33.890160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:33.890160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:33.890160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:33.890160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:33.890160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:33.890160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:33.890160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:33.890160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:33.892405 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:33.892405 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:33.892405 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:33.894360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 386/763] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:33.895588 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a588f-6797-4ea4-915c-93b7420ba61a None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:33.938845 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:33.972207 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baae7e7-5130-4008-af08-48b8a1c85bf4 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:34.014747 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c2911-fd11-4d88-9ed0-336968bdc62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:34.015796 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c2911-fd11-4d88-9ed0-336968bdc62a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:34.016288 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c2911-fd11-4d88-9ed0-336968bdc62a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:34.016751 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c2911-fd11-4d88-9ed0-336968bdc62a None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:34.017559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c2911-fd11-4d88-9ed0-336968bdc62a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:34.019366 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c2911-fd11-4d88-9ed0-336968bdc62a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:34.026210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 378/764] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Aug 8 16:32:33 2022] GET /identity/v3/services => generated 772 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:34.027403 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c2911-fd11-4d88-9ed0-336968bdc62a None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:34.069648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:34.087163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a588f-6797-4ea4-915c-93b7420ba61a None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:34.149886 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a97ab-e12d-40fd-b639-d3b7806fbab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:34.151374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a97ab-e12d-40fd-b639-d3b7806fbab9 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:34.151914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a97ab-e12d-40fd-b639-d3b7806fbab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:34.152403 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a97ab-e12d-40fd-b639-d3b7806fbab9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:34.168089 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a97ab-e12d-40fd-b639-d3b7806fbab9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:34.169976 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a97ab-e12d-40fd-b639-d3b7806fbab9 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:34.189486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-889a97ab-e12d-40fd-b639-d3b7806fbab9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:34.189486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:34.189486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:34.189486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:34.189486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:34.189486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:34.189486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:34.189486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:34.189486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:34.189486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:34.189486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:34.189486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:34.189486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:34.189486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:34.189486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:34.189486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:34.190654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:34.190654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:34.190654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:34.190654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:34.190654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:34.190654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:34.190654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:34.190654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:34.190654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:34.190654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:34.190654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:34.190654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:34.190654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:34.190654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:34.190654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:34.190654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:34.190654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:34.191831 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:34.191831 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:34.191831 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:34.194171 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 387/765] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:34.196335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943edaa1-3e74-4cee-8ee0-613221ede133 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:34.196973 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943edaa1-3e74-4cee-8ee0-613221ede133 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:34.197473 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943edaa1-3e74-4cee-8ee0-613221ede133 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:34.199458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 388/766] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:34.205070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943edaa1-3e74-4cee-8ee0-613221ede133 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:34.229882 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c2911-fd11-4d88-9ed0-336968bdc62a None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:34.287802 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f479b45c-fe4e-4461-820d-552c616a1e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:34.288419 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479b45c-fe4e-4461-820d-552c616a1e12 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:34.288694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479b45c-fe4e-4461-820d-552c616a1e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:34.288956 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479b45c-fe4e-4461-820d-552c616a1e12 None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:34.290195 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f479b45c-fe4e-4461-820d-552c616a1e12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:34.291199 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f479b45c-fe4e-4461-820d-552c616a1e12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:34.292639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f479b45c-fe4e-4461-820d-552c616a1e12 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:34.306348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 379/767] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:34.312193 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26542b51-f1e3-4eac-b174-97e901bdaed4 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:34.312610 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26542b51-f1e3-4eac-b174-97e901bdaed4 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:34.312748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26542b51-f1e3-4eac-b174-97e901bdaed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:34.331729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-26542b51-f1e3-4eac-b174-97e901bdaed4 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:34.332926 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-26542b51-f1e3-4eac-b174-97e901bdaed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:34.341119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:34.361439 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943edaa1-3e74-4cee-8ee0-613221ede133 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:34.414128 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01718030-e2d0-4edf-a579-3acae6855ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:34.415353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01718030-e2d0-4edf-a579-3acae6855ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:34.416279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01718030-e2d0-4edf-a579-3acae6855ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:34.416667 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01718030-e2d0-4edf-a579-3acae6855ee2 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:34.417649 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01718030-e2d0-4edf-a579-3acae6855ee2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:34.417955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01718030-e2d0-4edf-a579-3acae6855ee2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:34.420044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01718030-e2d0-4edf-a579-3acae6855ee2 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:34.436284 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 389/768] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:34.437192 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01718030-e2d0-4edf-a579-3acae6855ee2 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:34.445793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 380/769] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:32:34.452708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26542b51-f1e3-4eac-b174-97e901bdaed4 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:34.474954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:34.595607 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01718030-e2d0-4edf-a579-3acae6855ee2 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:34.604493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26542b51-f1e3-4eac-b174-97e901bdaed4 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:34.638434 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea79706-95cf-4046-be36-3938ccdfa4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:34.639535 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea79706-95cf-4046-be36-3938ccdfa4fa None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:34.640053 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea79706-95cf-4046-be36-3938ccdfa4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:34.640535 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea79706-95cf-4046-be36-3938ccdfa4fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:34.645542 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad9e06-23da-4fd6-ae10-8c2edb1479b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:34.646241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad9e06-23da-4fd6-ae10-8c2edb1479b4 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:34.646526 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad9e06-23da-4fd6-ae10-8c2edb1479b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:34.647006 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad9e06-23da-4fd6-ae10-8c2edb1479b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:34.650925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea79706-95cf-4046-be36-3938ccdfa4fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:34.652839 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea79706-95cf-4046-be36-3938ccdfa4fa None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:34.655239 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad9e06-23da-4fd6-ae10-8c2edb1479b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:34.656723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad9e06-23da-4fd6-ae10-8c2edb1479b4 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:34.667406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-eea79706-95cf-4046-be36-3938ccdfa4fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:34.667406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:34.667406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:34.667406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:34.667406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:34.667406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:34.667406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:34.667406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:34.667406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:34.667406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:34.667406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:34.667406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:34.667406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:34.667406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:34.667406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:34.668396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:34.668396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:34.668396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:34.668396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:34.668396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:34.668396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:34.668396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:34.668396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:34.668396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:34.668396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:34.668396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:34.668396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:34.668396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:34.668396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:34.668396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:34.668396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:34.668396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:34.669368 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:34.669368 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:34.669368 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:34.669368 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:34.669368 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:34.669368 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:34.669368 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:34.669368 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:34.670304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-edad9e06-23da-4fd6-ae10-8c2edb1479b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:34.670304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:34.670304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:34.670304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:34.670304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:34.670304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:34.670304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:34.670304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:34.670304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:34.670304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:34.670304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:34.670304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:34.670304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:34.670304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:34.670304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:34.671198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:34.671198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:34.671198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:34.671198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:34.671198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:34.671198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:34.671198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:34.671198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:34.671198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:34.671198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:34.671198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:34.671198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:34.671198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:34.671198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:34.671198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:34.671198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:34.671198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:34.672132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:34.672132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:34.672132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:34.672132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:34.672132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:34.672132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:34.672132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:34.672132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:34.672132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 381/770] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:34.672639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad9e06-23da-4fd6-ae10-8c2edb1479b4 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:34.674946 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 390/771] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:34.681449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea79706-95cf-4046-be36-3938ccdfa4fa None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:34.703034 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:34.771003 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:34.817627 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad9e06-23da-4fd6-ae10-8c2edb1479b4 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:34.869015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a086a-2888-4a9a-988c-69214c8b09e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:34.870476 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a086a-2888-4a9a-988c-69214c8b09e7 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:34.871248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a086a-2888-4a9a-988c-69214c8b09e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:34.871960 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a086a-2888-4a9a-988c-69214c8b09e7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:34.873237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a086a-2888-4a9a-988c-69214c8b09e7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:34.876077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a086a-2888-4a9a-988c-69214c8b09e7 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:34.879072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea79706-95cf-4046-be36-3938ccdfa4fa None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:34.892914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-966a086a-2888-4a9a-988c-69214c8b09e7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 16:32:34.892914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:34.892914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:32:34.892914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:32:34.892914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:32:34.892914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:32:34.892914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:32:34.892914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:34.892914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 16:32:34.892914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:34.892914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:32:34.892914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:34.892914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:34.892914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:34.892914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:34.892914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:34.892914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:34.894909 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:34.894909 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:34.894909 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:34.894909 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:34.894909 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:34.894909 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:34.894909 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:32:34.894909 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:32:34.894909 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:32:34.894909 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:32:34.894909 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:34.894909 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:34.894909 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:34.894909 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:34.894909 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:34.894909 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:34.896455 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:34.896455 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:34.896455 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:34.896455 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:34.896455 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:34.896455 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:34.896455 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:34.896455 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:34.896455 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:34.896455 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:34.896455 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:32:34.896455 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:34.896455 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 16:32:34.896455 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:34.896455 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 382/772] 10.222.0.13 () {56 vars in 1129 bytes} [Mon Aug 8 16:32:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:34.897752 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a086a-2888-4a9a-988c-69214c8b09e7 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:34.939997 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e34daf-370d-43fd-b3fa-994245ec11a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:34.941133 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e34daf-370d-43fd-b3fa-994245ec11a4 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:34.941717 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e34daf-370d-43fd-b3fa-994245ec11a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:34.942174 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e34daf-370d-43fd-b3fa-994245ec11a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:34.943276 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e34daf-370d-43fd-b3fa-994245ec11a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:34.944138 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e34daf-370d-43fd-b3fa-994245ec11a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:34.946432 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e34daf-370d-43fd-b3fa-994245ec11a4 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:34.965133 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 391/773] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:34.966303 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e34daf-370d-43fd-b3fa-994245ec11a4 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:35.063045 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:35.111010 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a086a-2888-4a9a-988c-69214c8b09e7 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:35.117918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e34daf-370d-43fd-b3fa-994245ec11a4 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:35.165238 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86920beb-a92d-481b-a39b-d31454d85157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:35.166473 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86920beb-a92d-481b-a39b-d31454d85157 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:35.166971 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86920beb-a92d-481b-a39b-d31454d85157 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:35.167449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86920beb-a92d-481b-a39b-d31454d85157 None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:35.168480 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:35.169349 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86920beb-a92d-481b-a39b-d31454d85157 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:35.170365 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86920beb-a92d-481b-a39b-d31454d85157 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:35.172221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86920beb-a92d-481b-a39b-d31454d85157 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:35.177964 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfbd89-f3e0-4849-940f-bcb5b60e14a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:35.178587 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 392/774] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Aug 8 16:32:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:35.179202 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfbd89-f3e0-4849-940f-bcb5b60e14a6 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:35.179796 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfbd89-f3e0-4849-940f-bcb5b60e14a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:35.179956 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86920beb-a92d-481b-a39b-d31454d85157 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:35.180274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfbd89-f3e0-4849-940f-bcb5b60e14a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:35.181570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfbd89-f3e0-4849-940f-bcb5b60e14a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:35.186036 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfbd89-f3e0-4849-940f-bcb5b60e14a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:35.186036 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfbd89-f3e0-4849-940f-bcb5b60e14a6 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:35.218041 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 383/775] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:35.218041 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfbd89-f3e0-4849-940f-bcb5b60e14a6 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:35.324118 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86920beb-a92d-481b-a39b-d31454d85157 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:35.370452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:35.377873 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5a7f5-56e2-4209-a336-3b5715faf009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:35.378919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5a7f5-56e2-4209-a336-3b5715faf009 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:35.379336 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5a7f5-56e2-4209-a336-3b5715faf009 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:35.379741 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5a7f5-56e2-4209-a336-3b5715faf009 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:35.380908 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5a7f5-56e2-4209-a336-3b5715faf009 None admin] RBAC: Adding query filter params (role.id=7764573472d34ecc8bedf7a9c7935170, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=9646662b0b3048f79ad5d2b9602f10a6) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:35.381354 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5a7f5-56e2-4209-a336-3b5715faf009 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:35.384058 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5a7f5-56e2-4209-a336-3b5715faf009 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:35.398647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 393/776] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:32:35 2022] GET /identity/v3/role_assignments?user.id=9646662b0b3048f79ad5d2b9602f10a6&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=7764573472d34ecc8bedf7a9c7935170 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:35.398953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfbd89-f3e0-4849-940f-bcb5b60e14a6 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:35.399566 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5a7f5-56e2-4209-a336-3b5715faf009 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:35.463652 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afc22d-7671-4677-8f5d-a6eaf4eeff9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:35.464677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afc22d-7671-4677-8f5d-a6eaf4eeff9c None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:35.465190 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afc22d-7671-4677-8f5d-a6eaf4eeff9c None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:35.465798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afc22d-7671-4677-8f5d-a6eaf4eeff9c None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:35.467138 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55afc22d-7671-4677-8f5d-a6eaf4eeff9c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:35.467972 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55afc22d-7671-4677-8f5d-a6eaf4eeff9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:35.470809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55afc22d-7671-4677-8f5d-a6eaf4eeff9c None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:35.485787 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 384/777] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Aug 8 16:32:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:35.485787 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea47a40-7ca8-49dc-a0d7-cf255d7731e4 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:35.485787 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea47a40-7ca8-49dc-a0d7-cf255d7731e4 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:35.485787 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea47a40-7ca8-49dc-a0d7-cf255d7731e4 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:35.487712 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 385/778] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:35.496089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea47a40-7ca8-49dc-a0d7-cf255d7731e4 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:35.583835 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:35.597179 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5a7f5-56e2-4209-a336-3b5715faf009 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:35.664088 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83714466-1b57-43f5-9388-c63bcfbbe434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:35.665900 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83714466-1b57-43f5-9388-c63bcfbbe434 None admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:35.666422 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83714466-1b57-43f5-9388-c63bcfbbe434 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:35.667073 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83714466-1b57-43f5-9388-c63bcfbbe434 None admin] PATH_INFO: `/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/9daa7489bced467984d39e8b8ac49d13/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:35.670937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83714466-1b57-43f5-9388-c63bcfbbe434 None admin] RBAC: Authorizing `identity:create_grant(project_id=56ab2fc79d964f4d86edefb4d1b2c993, user_id=9daa7489bced467984d39e8b8ac49d13, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:35.673749 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83714466-1b57-43f5-9388-c63bcfbbe434 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:35.685114 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea47a40-7ca8-49dc-a0d7-cf255d7731e4 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:35.741272 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ba9bb-139f-4ee1-9a53-e1241a21adaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:35.742233 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ba9bb-139f-4ee1-9a53-e1241a21adaf None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:35.743571 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ba9bb-139f-4ee1-9a53-e1241a21adaf None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:35.744130 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ba9bb-139f-4ee1-9a53-e1241a21adaf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:35.745364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ba9bb-139f-4ee1-9a53-e1241a21adaf None admin] RBAC: Adding query filter params (interface=public, service_id=fa6ec33d57fb4c52a4026dd4486fa426, region_id=RegionOne) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:35.745982 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ba9bb-139f-4ee1-9a53-e1241a21adaf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:35.748308 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ba9bb-139f-4ee1-9a53-e1241a21adaf None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:35.759570 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 394/779] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:32:35 2022] PUT /identity/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/9daa7489bced467984d39e8b8ac49d13/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:32:35.760569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 386/780] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Aug 8 16:32:35 2022] GET /identity/v3/endpoints?service_id=fa6ec33d57fb4c52a4026dd4486fa426&interface=public®ion_id=RegionOne => generated 194 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:35.770711 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa1e26-7167-429e-a72f-5cb34199e911 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:35.770711 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa1e26-7167-429e-a72f-5cb34199e911 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:35.770711 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa1e26-7167-429e-a72f-5cb34199e911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:35.775399 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ba9bb-139f-4ee1-9a53-e1241a21adaf None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:35.806617 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-b4fa1e26-7167-429e-a72f-5cb34199e911 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:35.806617 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b4fa1e26-7167-429e-a72f-5cb34199e911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:35.885525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:35.917234 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 395/781] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:32:35.950320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa1e26-7167-429e-a72f-5cb34199e911 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:36.005899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ba9bb-139f-4ee1-9a53-e1241a21adaf None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:36.054947 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:36.083663 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba11d30-d7a1-4ae3-9981-fbe0f233a63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:36.085265 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba11d30-d7a1-4ae3-9981-fbe0f233a63c None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:36.085265 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba11d30-d7a1-4ae3-9981-fbe0f233a63c None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:36.085265 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba11d30-d7a1-4ae3-9981-fbe0f233a63c None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:36.085932 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba11d30-d7a1-4ae3-9981-fbe0f233a63c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:36.088234 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba11d30-d7a1-4ae3-9981-fbe0f233a63c None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:36.095571 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 387/782] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Aug 8 16:32:35 2022] GET /identity/v3/services => generated 772 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:32:36.129359 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa1e26-7167-429e-a72f-5cb34199e911 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:36.175539 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f741f-aa33-4311-b977-945b6c67b29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:36.176515 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f741f-aa33-4311-b977-945b6c67b29a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:36.176946 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f741f-aa33-4311-b977-945b6c67b29a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:36.177360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f741f-aa33-4311-b977-945b6c67b29a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:36.191041 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f741f-aa33-4311-b977-945b6c67b29a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:32:36.196911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-3b4f741f-aa33-4311-b977-945b6c67b29a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:32:36.196911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:36.196911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:36.196911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:36.196911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:36.196911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:36.196911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:36.196911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:36.196911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:36.196911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:36.196911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:36.196911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:36.196911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:32:36.196911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:32:36.196911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:32:36.196911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:32:36.198375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:32:36.198375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:32:36.198375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:32:36.198375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:32:36.198375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:36.198375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:36.198375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:36.198375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:36.198375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:36.198375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:36.198375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:36.198375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:36.198375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:36.198375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:36.198375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:36.198375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:36.199452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:36.199452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:36.199452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:36.199452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:36.199452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:32:36.199452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:32:36.199452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:32:36.199452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:32:36.199452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:32:36.199452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:32:36.199452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:32:36.199452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:36.202010 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 396/783] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:32:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:32:36.208472 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba11d30-d7a1-4ae3-9981-fbe0f233a63c None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:36.365026 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba11d30-d7a1-4ae3-9981-fbe0f233a63c None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:36.417385 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34bbd0e-cc96-401f-8fd5-bdfc34d46fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:36.418007 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34bbd0e-cc96-401f-8fd5-bdfc34d46fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:36.418282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34bbd0e-cc96-401f-8fd5-bdfc34d46fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:36.418546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34bbd0e-cc96-401f-8fd5-bdfc34d46fb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:36.419448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34bbd0e-cc96-401f-8fd5-bdfc34d46fb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:36.419752 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34bbd0e-cc96-401f-8fd5-bdfc34d46fb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:36.421189 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34bbd0e-cc96-401f-8fd5-bdfc34d46fb9 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:36.435343 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 388/784] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:32:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:36.440877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f741f-aa33-4311-b977-945b6c67b29a None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:36.509820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:36.597839 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f741f-aa33-4311-b977-945b6c67b29a None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:36.649609 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8907dc8-4cdd-40ec-916f-03c8c4a5511f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:36.650705 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8907dc8-4cdd-40ec-916f-03c8c4a5511f None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:36.651229 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8907dc8-4cdd-40ec-916f-03c8c4a5511f None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:36.651735 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8907dc8-4cdd-40ec-916f-03c8c4a5511f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:36.672889 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8907dc8-4cdd-40ec-916f-03c8c4a5511f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:36.674782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8907dc8-4cdd-40ec-916f-03c8c4a5511f None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:36.697721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-e8907dc8-4cdd-40ec-916f-03c8c4a5511f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:36.697721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:36.697721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:36.697721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:36.697721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:36.697721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:36.697721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:36.697721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:36.697721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:36.697721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:36.697721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:36.697721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:36.697721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:36.697721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:36.697721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:36.697721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:36.698826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:36.698826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:36.698826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:36.698826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:36.698826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:36.698826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:36.698826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:36.698826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:36.698826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:36.698826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:36.698826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:36.698826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:36.698826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:36.698826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:36.698826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:36.698826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:36.698826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:36.699739 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:36.699739 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:36.699739 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:36.701699 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 397/785] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:36.707530 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34bbd0e-cc96-401f-8fd5-bdfc34d46fb9 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:36.790139 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:36.865775 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34bbd0e-cc96-401f-8fd5-bdfc34d46fb9 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:36.884959 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100f4bd-9849-4bcd-86b2-a7908b4d10d1 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:36.885616 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100f4bd-9849-4bcd-86b2-a7908b4d10d1 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:36.886545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100f4bd-9849-4bcd-86b2-a7908b4d10d1 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:36.888281 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 398/786] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:36.896441 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a39703-7e08-4705-877a-98d8c04db3ce None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:36.896789 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a39703-7e08-4705-877a-98d8c04db3ce None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:36.897059 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a39703-7e08-4705-877a-98d8c04db3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:36.927604 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d475d5-1e72-462a-9ed7-d55970a6e4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:36.928580 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d475d5-1e72-462a-9ed7-d55970a6e4ed None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:36.929073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d475d5-1e72-462a-9ed7-d55970a6e4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:36.929553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d475d5-1e72-462a-9ed7-d55970a6e4ed None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:36.931380 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d475d5-1e72-462a-9ed7-d55970a6e4ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:36.932652 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-84a39703-7e08-4705-877a-98d8c04db3ce None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:36.933226 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d475d5-1e72-462a-9ed7-d55970a6e4ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:36.934908 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d475d5-1e72-462a-9ed7-d55970a6e4ed None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:36.936549 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-84a39703-7e08-4705-877a-98d8c04db3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:36.963195 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 389/787] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:36.974614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d475d5-1e72-462a-9ed7-d55970a6e4ed None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:37.047345 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 399/788] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:32:37.088146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a39703-7e08-4705-877a-98d8c04db3ce None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:37.154954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:37.210394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d475d5-1e72-462a-9ed7-d55970a6e4ed None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:37.243299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:37.298480 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4ed88-3703-49d6-a7cc-2c201b64285a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:37.299581 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4ed88-3703-49d6-a7cc-2c201b64285a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:37.300070 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4ed88-3703-49d6-a7cc-2c201b64285a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:37.300555 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4ed88-3703-49d6-a7cc-2c201b64285a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:37.319172 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec4ed88-3703-49d6-a7cc-2c201b64285a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:37.323156 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec4ed88-3703-49d6-a7cc-2c201b64285a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:37.351729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-cec4ed88-3703-49d6-a7cc-2c201b64285a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:32:37.351729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:37.351729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:37.351729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:37.351729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:37.351729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:37.351729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:37.351729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:37.351729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:37.351729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:37.351729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:37.351729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:37.351729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:37.351729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:37.351729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:37.351729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:37.354203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:37.354203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:37.354203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:37.354203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:37.354203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:37.354203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:37.354203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:37.354203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:37.354203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:37.354203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:37.354203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:37.354203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:37.354203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:37.354203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:37.354203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:37.354203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:37.354203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:37.354203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:37.356967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:37.356967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:37.356967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:37.356967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:37.356967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:37.356967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:37.356967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:37.356967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:37.356967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:32:37.356967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:37.356967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 390/789] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:32:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:37.368176 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4ed88-3703-49d6-a7cc-2c201b64285a None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:37.371678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a39703-7e08-4705-877a-98d8c04db3ce None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:37.414610 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a2b2d-edf5-43f9-b288-0ef10c9306ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:37.416489 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a2b2d-edf5-43f9-b288-0ef10c9306ad None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:37.416489 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a2b2d-edf5-43f9-b288-0ef10c9306ad None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:37.416974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a2b2d-edf5-43f9-b288-0ef10c9306ad None admin] PATH_INFO: `/v3/services/network` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:37.418609 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a2b2d-edf5-43f9-b288-0ef10c9306ad None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:37.418892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a2b2d-edf5-43f9-b288-0ef10c9306ad None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:37.428320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-ad7a2b2d-edf5-43f9-b288-0ef10c9306ad None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 16:32:37.428320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:37.428320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:32:37.428320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:32:37.428320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:32:37.428320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:32:37.428320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:32:37.428320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:37.428320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 16:32:37.428320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:37.428320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:32:37.428320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:37.428320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:37.428320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:37.428320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:37.428320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:37.428320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:37.431465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:37.431465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:37.431465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:37.431465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:37.431465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:37.431465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:37.431465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:32:37.431465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:32:37.431465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:32:37.431465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:32:37.431465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:37.431465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:37.431465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:37.431465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:37.431465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:37.431465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:37.433250 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:37.433250 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:37.433250 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:37.433250 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:37.433250 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:37.433250 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:37.433250 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:37.433250 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:37.433250 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:37.433250 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:37.433250 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:32:37.433250 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:37.433250 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 16:32:37.433250 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:37.433250 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 400/790] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:37 2022] GET /identity/v3/services/network => generated 88 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:32:37.448612 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a2b2d-edf5-43f9-b288-0ef10c9306ad None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:37.635899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:37.646809 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a2b2d-edf5-43f9-b288-0ef10c9306ad None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:37.662751 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4ed88-3703-49d6-a7cc-2c201b64285a None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:37.687000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b28a2e-4bc9-403d-9837-2005a0049397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:37.687631 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b28a2e-4bc9-403d-9837-2005a0049397 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:37.687903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b28a2e-4bc9-403d-9837-2005a0049397 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:37.688178 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b28a2e-4bc9-403d-9837-2005a0049397 None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:37.688826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b28a2e-4bc9-403d-9837-2005a0049397 None admin] RBAC: Adding query filter params (name=network) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:37.689132 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b28a2e-4bc9-403d-9837-2005a0049397 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:37.690551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b28a2e-4bc9-403d-9837-2005a0049397 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:37.702641 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 401/791] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Aug 8 16:32:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:32:37.716538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b28a2e-4bc9-403d-9837-2005a0049397 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:37.755750 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08cf8f-2fa5-46ec-bf67-f7270cf4297f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:37.760045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08cf8f-2fa5-46ec-bf67-f7270cf4297f None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:37.761225 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08cf8f-2fa5-46ec-bf67-f7270cf4297f None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:37.763133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08cf8f-2fa5-46ec-bf67-f7270cf4297f None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:37.764757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08cf8f-2fa5-46ec-bf67-f7270cf4297f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:37.766176 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08cf8f-2fa5-46ec-bf67-f7270cf4297f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:37.773494 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08cf8f-2fa5-46ec-bf67-f7270cf4297f None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:37.796800 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 391/792] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Aug 8 16:32:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:37.802424 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08cf8f-2fa5-46ec-bf67-f7270cf4297f None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:37.858373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:37.976268 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08cf8f-2fa5-46ec-bf67-f7270cf4297f None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:38.001163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b28a2e-4bc9-403d-9837-2005a0049397 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:38.032277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22892f-b4f6-46ff-a84d-012fe9a91d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:38.033833 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22892f-b4f6-46ff-a84d-012fe9a91d28 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:38.034626 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22892f-b4f6-46ff-a84d-012fe9a91d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:38.035417 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22892f-b4f6-46ff-a84d-012fe9a91d28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:38.069933 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f22892f-b4f6-46ff-a84d-012fe9a91d28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:38.073746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f22892f-b4f6-46ff-a84d-012fe9a91d28 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:38.087514 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1eaa27-fbf1-4a97-bb4f-fb10ba7b84c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:38.089250 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1eaa27-fbf1-4a97-bb4f-fb10ba7b84c9 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:38.089867 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1eaa27-fbf1-4a97-bb4f-fb10ba7b84c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:38.090436 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1eaa27-fbf1-4a97-bb4f-fb10ba7b84c9 None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:38.091769 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1eaa27-fbf1-4a97-bb4f-fb10ba7b84c9 None admin] RBAC: Adding query filter params (type=network) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:38.092635 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1eaa27-fbf1-4a97-bb4f-fb10ba7b84c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:38.095258 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1eaa27-fbf1-4a97-bb4f-fb10ba7b84c9 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:38.105792 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 402/793] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Aug 8 16:32:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:32:38.114902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1eaa27-fbf1-4a97-bb4f-fb10ba7b84c9 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:38.132182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-5f22892f-b4f6-46ff-a84d-012fe9a91d28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:38.132182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:38.132182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:38.132182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:38.132182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:38.132182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:38.132182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:38.132182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:38.132182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:38.132182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:38.132182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:38.132182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:38.132182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:38.132182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:38.132182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:38.132182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:38.134274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:38.134274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:38.134274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:38.134274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:38.134274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:38.134274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:38.134274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:38.134274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:38.134274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:38.134274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:38.134274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:38.134274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:38.134274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:38.134274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:38.134274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:38.134274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:38.134274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:38.136332 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:38.136332 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:38.136332 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:38.139018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 392/794] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:38.145412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22892f-b4f6-46ff-a84d-012fe9a91d28 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:38.157150 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:38.371972 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:38.399963 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22892f-b4f6-46ff-a84d-012fe9a91d28 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:38.404948 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1eaa27-fbf1-4a97-bb4f-fb10ba7b84c9 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:38.452257 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a9d6d-c469-40d5-b480-e361770ba193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:38.452883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a9d6d-c469-40d5-b480-e361770ba193 None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:38.453175 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a9d6d-c469-40d5-b480-e361770ba193 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:38.453443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a9d6d-c469-40d5-b480-e361770ba193 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:38.454291 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a9d6d-c469-40d5-b480-e361770ba193 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:38.455674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a9d6d-c469-40d5-b480-e361770ba193 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:38.471874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20dc850-a9a1-49f8-992a-d5900e741954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:38.473047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dc850-a9a1-49f8-992a-d5900e741954 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:38.473722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dc850-a9a1-49f8-992a-d5900e741954 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:38.474272 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dc850-a9a1-49f8-992a-d5900e741954 None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:38.475529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20dc850-a9a1-49f8-992a-d5900e741954 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:38.476117 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20dc850-a9a1-49f8-992a-d5900e741954 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:38.478917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20dc850-a9a1-49f8-992a-d5900e741954 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:38.506789 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 393/795] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:32:38.506789 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c6275-6e54-43a5-b7c9-bb35528d2854 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:38.506789 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c6275-6e54-43a5-b7c9-bb35528d2854 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:38.506789 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c6275-6e54-43a5-b7c9-bb35528d2854 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:38.508734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 394/796] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:38.510731 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c6275-6e54-43a5-b7c9-bb35528d2854 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:38.579151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 403/797] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:32:38 2022] POST /identity/v3/endpoints => generated 337 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:32:38.581689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d095d-330b-48a0-aa12-4422e5075f30 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:38.582228 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d095d-330b-48a0-aa12-4422e5075f30 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:38.582817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d095d-330b-48a0-aa12-4422e5075f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:38.647887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-798d095d-330b-48a0-aa12-4422e5075f30 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:38.649987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-798d095d-330b-48a0-aa12-4422e5075f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:38.677176 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c6275-6e54-43a5-b7c9-bb35528d2854 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:38.719724 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:38.726263 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d8e1e-cf0f-4ceb-ae60-2c4c0dcb6b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:38.728070 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d8e1e-cf0f-4ceb-ae60-2c4c0dcb6b46 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:38.728569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d8e1e-cf0f-4ceb-ae60-2c4c0dcb6b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:38.729013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d8e1e-cf0f-4ceb-ae60-2c4c0dcb6b46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:38.738398 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d8e1e-cf0f-4ceb-ae60-2c4c0dcb6b46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:38.740467 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d8e1e-cf0f-4ceb-ae60-2c4c0dcb6b46 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:38.757877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-342d8e1e-cf0f-4ceb-ae60-2c4c0dcb6b46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:38.757877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:38.757877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:38.757877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:38.757877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:38.757877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:38.757877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:38.757877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:38.757877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:38.757877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:38.757877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:38.757877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:38.757877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:38.757877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:38.757877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:38.759217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:38.759217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:38.759217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:38.759217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:38.759217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:38.759217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:38.759217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:38.759217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:38.759217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:38.759217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:38.759217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:38.759217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:38.759217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:38.759217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:38.759217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:38.759217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:38.759217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:38.760331 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:38.760331 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:38.760331 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:38.760331 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:38.760331 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:38.760331 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:38.760331 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:38.760331 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:38.760804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 395/798] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:38.767320 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53b01b-5f4a-4d23-a594-7720a5402a5e None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:38.767320 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53b01b-5f4a-4d23-a594-7720a5402a5e None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:38.767320 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53b01b-5f4a-4d23-a594-7720a5402a5e None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:38.770126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 396/799] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:38.771888 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53b01b-5f4a-4d23-a594-7720a5402a5e None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:38.823359 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 404/800] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:32:38.832512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a599c-919d-4f2c-8eab-1e1f8dd7eaa7 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:38.833126 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a599c-919d-4f2c-8eab-1e1f8dd7eaa7 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:38.833614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a599c-919d-4f2c-8eab-1e1f8dd7eaa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:38.841598 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:38.864580 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-da7a599c-919d-4f2c-8eab-1e1f8dd7eaa7 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:38.866298 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-da7a599c-919d-4f2c-8eab-1e1f8dd7eaa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:38.903979 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53b01b-5f4a-4d23-a594-7720a5402a5e None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:38.946505 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004e42b-10ed-4eff-9a96-eff34f7fe48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:38.947618 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004e42b-10ed-4eff-9a96-eff34f7fe48b None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:38.948080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004e42b-10ed-4eff-9a96-eff34f7fe48b None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:38.948542 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004e42b-10ed-4eff-9a96-eff34f7fe48b None admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:38.949754 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004e42b-10ed-4eff-9a96-eff34f7fe48b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:38.950244 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004e42b-10ed-4eff-9a96-eff34f7fe48b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:38.952991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004e42b-10ed-4eff-9a96-eff34f7fe48b None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:38.981392 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 397/801] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:38.983575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004e42b-10ed-4eff-9a96-eff34f7fe48b None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:38.997116 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 405/802] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:32:38.998156 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7a599c-919d-4f2c-8eab-1e1f8dd7eaa7 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:39.098902 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:39.146009 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7a599c-919d-4f2c-8eab-1e1f8dd7eaa7 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:39.186639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004e42b-10ed-4eff-9a96-eff34f7fe48b None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:39.209054 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3544b4-a735-49dc-a951-4b7449ae096e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:39.210171 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3544b4-a735-49dc-a951-4b7449ae096e None admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:39.210680 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3544b4-a735-49dc-a951-4b7449ae096e None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:39.211154 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3544b4-a735-49dc-a951-4b7449ae096e None admin] PATH_INFO: `/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/c606decb832444e38a194b98f8d15f8c/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:39.214549 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3544b4-a735-49dc-a951-4b7449ae096e None admin] RBAC: Authorizing `identity:create_grant(project_id=56ab2fc79d964f4d86edefb4d1b2c993, user_id=c606decb832444e38a194b98f8d15f8c, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:39.217567 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3544b4-a735-49dc-a951-4b7449ae096e None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:39.221855 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:39.250182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41787316-75b1-4ae4-b58d-dd464c9734aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:39.251300 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41787316-75b1-4ae4-b58d-dd464c9734aa None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:39.251791 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41787316-75b1-4ae4-b58d-dd464c9734aa None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:39.252380 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41787316-75b1-4ae4-b58d-dd464c9734aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:39.270789 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41787316-75b1-4ae4-b58d-dd464c9734aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:32:39.277900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-41787316-75b1-4ae4-b58d-dd464c9734aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:39.277900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:39.277900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:39.277900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:39.277900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:39.277900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:39.277900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:39.277900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:39.277900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:39.277900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:39.277900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:39.277900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:39.277900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:32:39.277900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:32:39.277900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:32:39.277900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:32:39.279798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:32:39.279798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:32:39.279798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:32:39.279798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:32:39.279798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:39.279798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:39.279798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:39.279798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:39.279798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:39.279798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:39.279798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:39.279798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:39.279798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:39.279798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:39.279798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:39.279798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:39.281916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:39.281916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:39.281916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:39.281916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:39.281916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:32:39.281916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:32:39.281916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:32:39.281916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:32:39.281916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:32:39.281916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:32:39.281916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:39.281916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:39.281916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 398/803] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:32:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:39.281916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41787316-75b1-4ae4-b58d-dd464c9734aa None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:39.298912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 406/804] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:32:38 2022] PUT /identity/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/c606decb832444e38a194b98f8d15f8c/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:32:39.301059 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed9609-9a63-44ba-8289-b91ed8dcaf08 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:39.301541 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed9609-9a63-44ba-8289-b91ed8dcaf08 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:39.302031 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed9609-9a63-44ba-8289-b91ed8dcaf08 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:39.304366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 407/805] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:39.306959 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed9609-9a63-44ba-8289-b91ed8dcaf08 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:39.472010 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41787316-75b1-4ae4-b58d-dd464c9734aa None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:39.484037 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed9609-9a63-44ba-8289-b91ed8dcaf08 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:39.497276 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:39.529573 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90204333-6fe1-4326-8fea-052f88820a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:39.530986 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90204333-6fe1-4326-8fea-052f88820a52 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:39.531631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90204333-6fe1-4326-8fea-052f88820a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:39.532202 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90204333-6fe1-4326-8fea-052f88820a52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:39.546814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960215b5-aaae-4a55-8399-d6f0c71375e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:39.547690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960215b5-aaae-4a55-8399-d6f0c71375e8 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:39.548142 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960215b5-aaae-4a55-8399-d6f0c71375e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:39.548591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960215b5-aaae-4a55-8399-d6f0c71375e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:39.549696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960215b5-aaae-4a55-8399-d6f0c71375e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:39.550197 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960215b5-aaae-4a55-8399-d6f0c71375e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:39.552652 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960215b5-aaae-4a55-8399-d6f0c71375e8 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:39.554067 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90204333-6fe1-4326-8fea-052f88820a52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:32:39.563745 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-90204333-6fe1-4326-8fea-052f88820a52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:32:39.563745 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:39.563745 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:39.563745 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:39.563745 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:39.563745 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:39.563745 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:39.563745 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:39.563745 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:39.563745 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:39.563745 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:39.563745 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:39.563745 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:32:39.563745 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:32:39.563745 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:32:39.563745 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:32:39.565603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:32:39.565603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:32:39.565603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:32:39.565603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:32:39.565603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:39.565603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:39.565603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:39.565603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:39.565603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:39.565603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:39.565603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:39.565603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:39.565603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:39.565603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:39.565603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:39.565603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:39.567927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:39.567927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:39.567927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:39.567927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:39.567927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:32:39.567927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:32:39.567927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:32:39.567927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:32:39.567927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:32:39.567927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:32:39.567927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:32:39.567927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:39.567927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 399/806] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:32:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:39.569282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2002e77-267e-462e-ac39-4a31b90960ff None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:39.569282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2002e77-267e-462e-ac39-4a31b90960ff None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:39.569282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2002e77-267e-462e-ac39-4a31b90960ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:39.576579 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 408/807] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Aug 8 16:32:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:39.578482 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960215b5-aaae-4a55-8399-d6f0c71375e8 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:39.597660 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-b2002e77-267e-462e-ac39-4a31b90960ff None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:39.599536 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-b2002e77-267e-462e-ac39-4a31b90960ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:39.678549 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:39.712305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 400/808] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:32:39.714274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2002e77-267e-462e-ac39-4a31b90960ff None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:39.747645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960215b5-aaae-4a55-8399-d6f0c71375e8 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:39.795743 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1259766-1800-4e4a-9f04-93365eb4c991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:39.796742 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1259766-1800-4e4a-9f04-93365eb4c991 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:39.796887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1259766-1800-4e4a-9f04-93365eb4c991 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:39.797117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1259766-1800-4e4a-9f04-93365eb4c991 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:39.798206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1259766-1800-4e4a-9f04-93365eb4c991 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:39.798771 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1259766-1800-4e4a-9f04-93365eb4c991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:39.801051 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1259766-1800-4e4a-9f04-93365eb4c991 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:39.815341 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 409/809] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:32:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:39.815341 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1259766-1800-4e4a-9f04-93365eb4c991 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:39.884747 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2002e77-267e-462e-ac39-4a31b90960ff None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:39.886607 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:39.924463 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a82f0-a58e-4404-a4b8-255c68f0b05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:39.925112 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a82f0-a58e-4404-a4b8-255c68f0b05c None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:39.925392 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a82f0-a58e-4404-a4b8-255c68f0b05c None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:39.925647 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a82f0-a58e-4404-a4b8-255c68f0b05c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:39.940589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a82f0-a58e-4404-a4b8-255c68f0b05c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:39.942046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a82f0-a58e-4404-a4b8-255c68f0b05c None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:39.958297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-514a82f0-a58e-4404-a4b8-255c68f0b05c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:39.958297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:39.958297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:39.958297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:39.958297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:39.958297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:39.958297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:39.958297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:39.958297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:39.958297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:39.958297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:39.958297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:39.958297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:39.958297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:39.958297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:39.958297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:39.959373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:39.959373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:39.959373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:39.959373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:39.959373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:39.959373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:39.959373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:39.959373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:39.959373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:39.959373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:39.959373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:39.959373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:39.959373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:39.959373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:39.959373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:39.959373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:39.959373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:39.960305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:39.960305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:39.960305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:39.960305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 401/810] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:39.960533 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a82f0-a58e-4404-a4b8-255c68f0b05c None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:39.989909 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1259766-1800-4e4a-9f04-93365eb4c991 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:40.013426 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:40.029271 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b1248-fe94-4d3a-901f-4daa6812f62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:40.030036 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b1248-fe94-4d3a-901f-4daa6812f62a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:40.030328 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b1248-fe94-4d3a-901f-4daa6812f62a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:40.030667 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b1248-fe94-4d3a-901f-4daa6812f62a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:40.031228 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b1248-fe94-4d3a-901f-4daa6812f62a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:40.032603 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b1248-fe94-4d3a-901f-4daa6812f62a None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:40.043086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-4a3b1248-fe94-4d3a-901f-4daa6812f62a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 16:32:40.043086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:40.043086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:32:40.043086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:32:40.043086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:32:40.043086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:32:40.043086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:32:40.043086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:40.043086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 16:32:40.043086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:40.043086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:32:40.043086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:40.043086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:40.043086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:40.043086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:40.043086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:40.043086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:40.053878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:40.053878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:40.053878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:40.053878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:40.053878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:40.053878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:40.053878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:32:40.053878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:32:40.053878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:32:40.053878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:32:40.053878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:40.053878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:40.053878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:40.053878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:40.053878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:40.053878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:40.056269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:40.056269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:40.056269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:40.056269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:40.056269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:40.056269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:40.056269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:40.056269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:40.056269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:40.056269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:40.056269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:32:40.056269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:40.056269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 16:32:40.056269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:40.056269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 410/811] 10.222.0.13 () {56 vars in 1129 bytes} [Mon Aug 8 16:32:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:40.058159 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b1248-fe94-4d3a-901f-4daa6812f62a None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:40.100101 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a82f0-a58e-4404-a4b8-255c68f0b05c None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:40.141481 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30262379-3236-4d23-90b7-6e868d34e30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:40.142121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30262379-3236-4d23-90b7-6e868d34e30f None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:40.142395 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30262379-3236-4d23-90b7-6e868d34e30f None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:40.142693 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30262379-3236-4d23-90b7-6e868d34e30f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:40.154966 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30262379-3236-4d23-90b7-6e868d34e30f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:40.156451 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30262379-3236-4d23-90b7-6e868d34e30f None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:40.164215 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b1248-fe94-4d3a-901f-4daa6812f62a None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:40.173049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-30262379-3236-4d23-90b7-6e868d34e30f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:40.173049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:40.173049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:40.173049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:40.173049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:40.173049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:40.173049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:40.173049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:40.173049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:40.173049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:40.173049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:40.173049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:40.173049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:40.173049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:40.173049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:40.173049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:40.174175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:40.174175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:40.174175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:40.174175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:40.174175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:40.174175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:40.174175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:40.174175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:40.174175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:40.174175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:40.174175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:40.174175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:40.174175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:40.174175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:40.174175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:40.174175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:40.174175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:40.181744 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:40.181744 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:40.181744 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:40.181744 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 402/812] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:40.181744 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30262379-3236-4d23-90b7-6e868d34e30f None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:40.189342 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:40.217978 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2283c1a-30e3-4d1e-97a0-5d44b5388f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:40.218622 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2283c1a-30e3-4d1e-97a0-5d44b5388f4c None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:40.218904 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2283c1a-30e3-4d1e-97a0-5d44b5388f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:40.219175 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2283c1a-30e3-4d1e-97a0-5d44b5388f4c None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:40.219832 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2283c1a-30e3-4d1e-97a0-5d44b5388f4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:40.220130 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2283c1a-30e3-4d1e-97a0-5d44b5388f4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:40.221507 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2283c1a-30e3-4d1e-97a0-5d44b5388f4c None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:40.245743 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 411/813] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:40.245743 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2283c1a-30e3-4d1e-97a0-5d44b5388f4c None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:40.309403 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:40.344231 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30262379-3236-4d23-90b7-6e868d34e30f None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:40.396648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2283c1a-30e3-4d1e-97a0-5d44b5388f4c None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:40.402961 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df0f26-d8d8-431b-8dcd-b8cd14afda68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:40.403904 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df0f26-d8d8-431b-8dcd-b8cd14afda68 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:40.404333 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df0f26-d8d8-431b-8dcd-b8cd14afda68 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:40.404750 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df0f26-d8d8-431b-8dcd-b8cd14afda68 None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:40.405760 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df0f26-d8d8-431b-8dcd-b8cd14afda68 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:40.406238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df0f26-d8d8-431b-8dcd-b8cd14afda68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:40.408622 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df0f26-d8d8-431b-8dcd-b8cd14afda68 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:40.421317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 403/814] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Aug 8 16:32:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:40.421317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df0f26-d8d8-431b-8dcd-b8cd14afda68 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:40.465001 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94159047-2a00-4d39-a17a-e4ae1b5bd01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:40.465895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94159047-2a00-4d39-a17a-e4ae1b5bd01f None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:40.466338 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94159047-2a00-4d39-a17a-e4ae1b5bd01f None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:40.466813 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94159047-2a00-4d39-a17a-e4ae1b5bd01f None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:40.467945 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94159047-2a00-4d39-a17a-e4ae1b5bd01f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:40.468433 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94159047-2a00-4d39-a17a-e4ae1b5bd01f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:40.470831 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94159047-2a00-4d39-a17a-e4ae1b5bd01f None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:40.494776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 412/815] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:40.494776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94159047-2a00-4d39-a17a-e4ae1b5bd01f None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:40.536167 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:40.572973 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df0f26-d8d8-431b-8dcd-b8cd14afda68 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:40.615987 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f8ad4-5b1e-46c3-93d3-f2c2598be385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:40.616622 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f8ad4-5b1e-46c3-93d3-f2c2598be385 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:40.616894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f8ad4-5b1e-46c3-93d3-f2c2598be385 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:40.617156 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f8ad4-5b1e-46c3-93d3-f2c2598be385 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:40.630302 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f8ad4-5b1e-46c3-93d3-f2c2598be385 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:40.632866 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f8ad4-5b1e-46c3-93d3-f2c2598be385 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:40.650725 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94159047-2a00-4d39-a17a-e4ae1b5bd01f None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:40.652493 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:40.656713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-7f8f8ad4-5b1e-46c3-93d3-f2c2598be385 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:32:40.656713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:40.656713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:40.656713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:40.656713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:40.656713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:40.656713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:40.656713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:40.656713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:40.656713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:40.656713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:40.656713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:40.656713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:40.656713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:40.656713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:40.656713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:40.657721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:40.657721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:40.657721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:40.657721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:40.657721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:40.657721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:40.657721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:40.657721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:40.657721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:40.657721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:40.657721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:40.657721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:40.657721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:40.657721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:40.657721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:40.657721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:40.657721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:40.657721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:40.658731 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:40.658731 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:40.658731 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:40.658731 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:40.658731 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:40.658731 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:40.658731 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:40.658731 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:40.658731 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:32:40.658731 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:40.658731 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 404/816] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Aug 8 16:32:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:40.660394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f8ad4-5b1e-46c3-93d3-f2c2598be385 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:40.693672 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758554c1-7704-4ca6-a6b8-26d8e4e5f177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:40.694658 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758554c1-7704-4ca6-a6b8-26d8e4e5f177 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:40.695157 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758554c1-7704-4ca6-a6b8-26d8e4e5f177 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:40.695579 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758554c1-7704-4ca6-a6b8-26d8e4e5f177 None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:40.697497 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758554c1-7704-4ca6-a6b8-26d8e4e5f177 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:40.698074 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758554c1-7704-4ca6-a6b8-26d8e4e5f177 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:40.700173 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758554c1-7704-4ca6-a6b8-26d8e4e5f177 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:40.710156 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 413/817] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Aug 8 16:32:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:40.710156 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758554c1-7704-4ca6-a6b8-26d8e4e5f177 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:40.823480 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758554c1-7704-4ca6-a6b8-26d8e4e5f177 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:40.849998 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f8ad4-5b1e-46c3-93d3-f2c2598be385 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:40.862938 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14cf7d-a1dc-4b83-96cf-8a7aadcf63af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:40.863560 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14cf7d-a1dc-4b83-96cf-8a7aadcf63af None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:40.863839 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14cf7d-a1dc-4b83-96cf-8a7aadcf63af None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:40.864107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14cf7d-a1dc-4b83-96cf-8a7aadcf63af None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:40.864842 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa14cf7d-a1dc-4b83-96cf-8a7aadcf63af None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:40.865145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa14cf7d-a1dc-4b83-96cf-8a7aadcf63af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:40.867228 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa14cf7d-a1dc-4b83-96cf-8a7aadcf63af None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:40.892932 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 414/818] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Aug 8 16:32:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:40.892932 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14cf7d-a1dc-4b83-96cf-8a7aadcf63af None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:40.904286 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:40.913362 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c691326-ff99-42e0-b10f-e9cfa4243e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:40.914441 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c691326-ff99-42e0-b10f-e9cfa4243e5e None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:40.914944 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c691326-ff99-42e0-b10f-e9cfa4243e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:40.915419 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c691326-ff99-42e0-b10f-e9cfa4243e5e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:40.924725 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:40.927929 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c691326-ff99-42e0-b10f-e9cfa4243e5e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:40.929391 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c691326-ff99-42e0-b10f-e9cfa4243e5e None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:40.944916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-6c691326-ff99-42e0-b10f-e9cfa4243e5e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:32:40.944916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:40.944916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:40.944916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:40.944916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:40.944916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:40.944916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:40.944916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:40.944916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:40.944916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:40.944916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:40.944916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:40.944916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:40.944916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:40.944916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:40.944916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:40.947373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:40.947373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:40.947373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:40.947373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:40.947373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:40.947373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:40.947373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:40.947373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:40.947373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:40.947373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:40.947373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:40.947373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:40.947373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:40.947373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:40.947373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:40.947373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:40.947373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:40.947373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:40.949212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:40.949212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:40.949212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:40.949212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:40.949212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:40.949212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:40.949212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:40.949212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:40.949212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:32:40.949212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:40.952059 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 405/819] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Aug 8 16:32:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:40.952827 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c691326-ff99-42e0-b10f-e9cfa4243e5e None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:41.042412 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14cf7d-a1dc-4b83-96cf-8a7aadcf63af None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:41.077710 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c691326-ff99-42e0-b10f-e9cfa4243e5e None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:41.108693 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a76b5-a867-4253-b9f7-6868d6362be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:41.109623 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a76b5-a867-4253-b9f7-6868d6362be9 None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:41.110008 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a76b5-a867-4253-b9f7-6868d6362be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:41.110382 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a76b5-a867-4253-b9f7-6868d6362be9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:41.111380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a76b5-a867-4253-b9f7-6868d6362be9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:41.113414 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a76b5-a867-4253-b9f7-6868d6362be9 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:41.120421 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06bc52-0ba4-40fe-8ff8-f692a7928a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:41.121349 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06bc52-0ba4-40fe-8ff8-f692a7928a25 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:41.121797 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06bc52-0ba4-40fe-8ff8-f692a7928a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:41.122255 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06bc52-0ba4-40fe-8ff8-f692a7928a25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:41.136626 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a06bc52-0ba4-40fe-8ff8-f692a7928a25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:41.138115 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a06bc52-0ba4-40fe-8ff8-f692a7928a25 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:41.156104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-1a06bc52-0ba4-40fe-8ff8-f692a7928a25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:41.156104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:41.156104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:41.156104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:41.156104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:41.156104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:41.156104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:41.156104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:41.156104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:41.156104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:41.156104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:41.156104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:41.156104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:41.156104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:41.156104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:41.156104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:41.158020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:41.158020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:41.158020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:41.158020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:41.158020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:41.158020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:41.158020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:41.158020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:41.158020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:41.158020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:41.158020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:41.158020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:41.158020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:41.158020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:41.158020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:41.158020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:41.158020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:41.160429 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:41.160429 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:41.160429 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:41.160429 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 406/820] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:41.160429 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06bc52-0ba4-40fe-8ff8-f692a7928a25 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:41.211151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 415/821] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:32:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:32:41.214407 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a76b5-a867-4253-b9f7-6868d6362be9 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:41.224719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:41.318175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06bc52-0ba4-40fe-8ff8-f692a7928a25 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:41.367522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:41.388327 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17d44c-d5eb-454d-b787-ac76295334e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:41.389736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17d44c-d5eb-454d-b787-ac76295334e1 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:41.390387 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17d44c-d5eb-454d-b787-ac76295334e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:41.391001 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17d44c-d5eb-454d-b787-ac76295334e1 None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:41.391952 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d17d44c-d5eb-454d-b787-ac76295334e1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:41.392439 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d17d44c-d5eb-454d-b787-ac76295334e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:41.394773 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d17d44c-d5eb-454d-b787-ac76295334e1 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:41.406202 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a76b5-a867-4253-b9f7-6868d6362be9 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:41.431596 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 407/822] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Aug 8 16:32:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:41.441543 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17d44c-d5eb-454d-b787-ac76295334e1 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:41.470263 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9e55e-072e-4df4-965d-f16cf2c1d95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:41.471373 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9e55e-072e-4df4-965d-f16cf2c1d95a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:41.471871 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9e55e-072e-4df4-965d-f16cf2c1d95a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:41.472359 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9e55e-072e-4df4-965d-f16cf2c1d95a None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:41.479340 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f9e55e-072e-4df4-965d-f16cf2c1d95a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:41.479340 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f9e55e-072e-4df4-965d-f16cf2c1d95a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:41.479340 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f9e55e-072e-4df4-965d-f16cf2c1d95a None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:41.501551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 416/823] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:41.512461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9e55e-072e-4df4-965d-f16cf2c1d95a None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:41.579081 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:41.590050 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17d44c-d5eb-454d-b787-ac76295334e1 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:41.677834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe589f2b-53c4-400c-8330-53f2d3381999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:41.679118 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe589f2b-53c4-400c-8330-53f2d3381999 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:41.680932 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe589f2b-53c4-400c-8330-53f2d3381999 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:41.681926 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe589f2b-53c4-400c-8330-53f2d3381999 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:41.687826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9e55e-072e-4df4-965d-f16cf2c1d95a None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:41.776274 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6437921-e1dc-4bb9-afc8-0d0d23878261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:41.777710 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6437921-e1dc-4bb9-afc8-0d0d23878261 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:41.778977 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6437921-e1dc-4bb9-afc8-0d0d23878261 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:41.779225 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6437921-e1dc-4bb9-afc8-0d0d23878261 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:41.794011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe589f2b-53c4-400c-8330-53f2d3381999 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:41.795608 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe589f2b-53c4-400c-8330-53f2d3381999 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:41.796708 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6437921-e1dc-4bb9-afc8-0d0d23878261 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:41.801474 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6437921-e1dc-4bb9-afc8-0d0d23878261 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:41.808602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:41.842670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-c6437921-e1dc-4bb9-afc8-0d0d23878261 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:41.842670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:41.842670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:41.842670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:41.842670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:41.842670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:41.842670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:41.842670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:41.842670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:41.842670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:41.842670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:41.842670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:41.842670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:41.842670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:41.842670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:41.844396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:41.844396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:41.844396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:41.844396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:41.844396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:41.844396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:41.844396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:41.844396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:41.844396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:41.844396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:41.844396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:41.844396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:41.844396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:41.844396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:41.844396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:41.844396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:41.844396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:41.846177 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:41.846177 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:41.846177 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:41.846177 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:41.846177 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:41.846177 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:41.846177 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:41.846177 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:41.846177 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 417/824] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:41.854461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6437921-e1dc-4bb9-afc8-0d0d23878261 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:41.969509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:41.987089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 408/825] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:32:41 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:32:41.992340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe589f2b-53c4-400c-8330-53f2d3381999 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:42.095520 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6437921-e1dc-4bb9-afc8-0d0d23878261 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:42.159211 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1eac4-a73e-419a-b19e-94c1f2136c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:42.159845 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1eac4-a73e-419a-b19e-94c1f2136c7a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:42.160109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1eac4-a73e-419a-b19e-94c1f2136c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:42.160466 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1eac4-a73e-419a-b19e-94c1f2136c7a None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:42.161178 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1eac4-a73e-419a-b19e-94c1f2136c7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:42.161487 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1eac4-a73e-419a-b19e-94c1f2136c7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:42.162976 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1eac4-a73e-419a-b19e-94c1f2136c7a None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:42.175757 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 418/826] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:42.177060 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79105789-908c-474a-9685-4a5c4013433d None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:42.177338 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79105789-908c-474a-9685-4a5c4013433d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:42.177610 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79105789-908c-474a-9685-4a5c4013433d None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:42.179173 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 419/827] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:42.185192 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4617da-2397-4278-816c-9ca78527793d None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:42.185407 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4617da-2397-4278-816c-9ca78527793d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:42.185675 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4617da-2397-4278-816c-9ca78527793d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:42.194153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe589f2b-53c4-400c-8330-53f2d3381999 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:42.205484 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-ca4617da-2397-4278-816c-9ca78527793d None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:42.206729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ca4617da-2397-4278-816c-9ca78527793d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:42.257948 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31613d-cea8-404a-a4e2-4c6642651db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:42.259825 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31613d-cea8-404a-a4e2-4c6642651db5 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:42.259825 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31613d-cea8-404a-a4e2-4c6642651db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:42.259825 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31613d-cea8-404a-a4e2-4c6642651db5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:42.271972 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31613d-cea8-404a-a4e2-4c6642651db5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:42.273444 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31613d-cea8-404a-a4e2-4c6642651db5 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:42.290834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-bb31613d-cea8-404a-a4e2-4c6642651db5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:42.290834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:42.290834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:42.290834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:42.290834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:42.290834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:42.290834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:42.290834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:42.290834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:42.290834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:42.290834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:42.290834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:42.290834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:42.290834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:42.290834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:42.290834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:42.292844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:42.292844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:42.292844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:42.292844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:42.292844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:42.292844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:42.292844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:42.292844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:42.292844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:42.292844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:42.292844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:42.292844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:42.292844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:42.292844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:42.292844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:42.292844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:42.292844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:42.294856 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:42.294856 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:42.294856 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:42.296756 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 409/828] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:32:42.297958 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31613d-cea8-404a-a4e2-4c6642651db5 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:42.298390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:42.313591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 420/829] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:32:42.314473 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4617da-2397-4278-816c-9ca78527793d None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:42.393793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:42.487157 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4617da-2397-4278-816c-9ca78527793d None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:42.492345 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31613d-cea8-404a-a4e2-4c6642651db5 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:42.541587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb5986-8fe2-4ce7-aa53-1a3a8b2fb873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:42.542467 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb5986-8fe2-4ce7-aa53-1a3a8b2fb873 None admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:42.542845 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb5986-8fe2-4ce7-aa53-1a3a8b2fb873 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:42.543203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb5986-8fe2-4ce7-aa53-1a3a8b2fb873 None admin] PATH_INFO: `/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/9646662b0b3048f79ad5d2b9602f10a6/roles/7764573472d34ecc8bedf7a9c7935170` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:42.557718 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd229fa-9fa2-4288-b370-bdb9b908b9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:42.560022 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd229fa-9fa2-4288-b370-bdb9b908b9f0 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:42.561138 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd229fa-9fa2-4288-b370-bdb9b908b9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:42.561814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd229fa-9fa2-4288-b370-bdb9b908b9f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:42.563252 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd229fa-9fa2-4288-b370-bdb9b908b9f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:42.564058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb5986-8fe2-4ce7-aa53-1a3a8b2fb873 None admin] RBAC: Authorizing `identity:create_grant(project_id=56ab2fc79d964f4d86edefb4d1b2c993, user_id=9646662b0b3048f79ad5d2b9602f10a6, role_id=7764573472d34ecc8bedf7a9c7935170)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:42.564621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd229fa-9fa2-4288-b370-bdb9b908b9f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:42.566723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb5986-8fe2-4ce7-aa53-1a3a8b2fb873 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:42.567880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd229fa-9fa2-4288-b370-bdb9b908b9f0 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:42.593489 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 421/830] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:42.595380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd229fa-9fa2-4288-b370-bdb9b908b9f0 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:42.646561 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 410/831] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:32:42 2022] PUT /identity/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/9646662b0b3048f79ad5d2b9602f10a6/roles/7764573472d34ecc8bedf7a9c7935170 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:32:42.648233 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb5986-8fe2-4ce7-aa53-1a3a8b2fb873 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:42.683896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:42.796383 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd229fa-9fa2-4288-b370-bdb9b908b9f0 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:42.860275 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07078a37-5a64-42d8-9b61-a305fa6569c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:42.861698 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07078a37-5a64-42d8-9b61-a305fa6569c2 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:42.862185 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07078a37-5a64-42d8-9b61-a305fa6569c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:42.863033 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07078a37-5a64-42d8-9b61-a305fa6569c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:42.866209 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:42.874182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb5986-8fe2-4ce7-aa53-1a3a8b2fb873 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:42.898542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07078a37-5a64-42d8-9b61-a305fa6569c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:32:42.909528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-07078a37-5a64-42d8-9b61-a305fa6569c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:32:42.909528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:42.909528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:42.909528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:42.909528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:42.909528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:42.909528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:42.909528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:42.909528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:42.909528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:42.909528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:42.909528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:42.909528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:32:42.909528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:32:42.909528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:32:42.909528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:32:42.910894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:32:42.910894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:32:42.910894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:32:42.910894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:32:42.910894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:42.910894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:42.910894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:42.910894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:42.910894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:42.910894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:42.910894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:42.910894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:42.910894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:42.910894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:42.910894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:42.910894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:42.911936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:42.911936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:42.911936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:42.911936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:42.911936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:32:42.911936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:32:42.911936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:32:42.911936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:32:42.911936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:32:42.911936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:32:42.911936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:32:42.911936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:42.921669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 422/832] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:32:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:42.926967 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07078a37-5a64-42d8-9b61-a305fa6569c2 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:42.947868 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd5d6a-2445-4f7d-91ea-6b1c828ee823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:42.949130 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd5d6a-2445-4f7d-91ea-6b1c828ee823 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:42.949575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd5d6a-2445-4f7d-91ea-6b1c828ee823 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:42.950023 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd5d6a-2445-4f7d-91ea-6b1c828ee823 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:42.968280 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd5d6a-2445-4f7d-91ea-6b1c828ee823 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:42.970196 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd5d6a-2445-4f7d-91ea-6b1c828ee823 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:42.984525 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-c6dd5d6a-2445-4f7d-91ea-6b1c828ee823 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:42.984525 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:42.984525 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:42.984525 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:42.984525 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:42.984525 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:42.984525 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:42.984525 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:42.984525 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:42.984525 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:42.984525 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:42.984525 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:42.984525 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:42.984525 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:42.984525 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:42.985569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:42.985569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:42.985569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:42.985569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:42.985569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:42.985569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:42.985569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:42.985569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:42.985569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:42.985569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:42.985569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:42.985569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:42.985569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:42.985569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:42.985569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:42.985569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:42.985569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:42.986528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:42.986528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:42.986528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:42.986528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:42.986528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:42.986528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:42.986528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:42.986528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:42.988543 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 411/833] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:43.003773 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd5d6a-2445-4f7d-91ea-6b1c828ee823 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:43.137289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07078a37-5a64-42d8-9b61-a305fa6569c2 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:43.147437 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd5d6a-2445-4f7d-91ea-6b1c828ee823 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:43.158014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:43.201460 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04f5cc-add1-4a81-bdcc-4dece3de0155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:43.202301 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04f5cc-add1-4a81-bdcc-4dece3de0155 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:43.202602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04f5cc-add1-4a81-bdcc-4dece3de0155 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:43.202868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04f5cc-add1-4a81-bdcc-4dece3de0155 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:43.203720 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd04f5cc-add1-4a81-bdcc-4dece3de0155 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:43.204002 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd04f5cc-add1-4a81-bdcc-4dece3de0155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:43.205165 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f633a3e-e15a-4102-b8c8-8096678d688f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:43.205308 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd04f5cc-add1-4a81-bdcc-4dece3de0155 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:43.205792 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f633a3e-e15a-4102-b8c8-8096678d688f None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:43.206146 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f633a3e-e15a-4102-b8c8-8096678d688f None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:43.206494 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f633a3e-e15a-4102-b8c8-8096678d688f None admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:43.207422 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f633a3e-e15a-4102-b8c8-8096678d688f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:43.207823 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f633a3e-e15a-4102-b8c8-8096678d688f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:43.209474 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f633a3e-e15a-4102-b8c8-8096678d688f None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:43.221723 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 423/834] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:32:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:32:43.222913 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04f5cc-add1-4a81-bdcc-4dece3de0155 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:43.228524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 412/835] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:32:43.234717 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f633a3e-e15a-4102-b8c8-8096678d688f None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:43.259135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:43.387577 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04f5cc-add1-4a81-bdcc-4dece3de0155 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:43.401197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f633a3e-e15a-4102-b8c8-8096678d688f None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:43.435462 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3202593-d47f-4574-a623-5cd8a2869fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:43.436245 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3202593-d47f-4574-a623-5cd8a2869fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:43.436606 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3202593-d47f-4574-a623-5cd8a2869fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:43.436883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3202593-d47f-4574-a623-5cd8a2869fc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:43.450410 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3202593-d47f-4574-a623-5cd8a2869fc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:43.451973 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3202593-d47f-4574-a623-5cd8a2869fc5 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:43.459833 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc146f-01a0-494e-8194-0628e478f2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:43.461720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc146f-01a0-494e-8194-0628e478f2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:43.462298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc146f-01a0-494e-8194-0628e478f2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:43.462802 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc146f-01a0-494e-8194-0628e478f2a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:43.464701 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc146f-01a0-494e-8194-0628e478f2a6 None admin] RBAC: Adding query filter params (role.id=e578ef831127463d9b48cc89fe20329c, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=9daa7489bced467984d39e8b8ac49d13) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:43.465547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc146f-01a0-494e-8194-0628e478f2a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:43.467579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc146f-01a0-494e-8194-0628e478f2a6 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:43.472039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-c3202593-d47f-4574-a623-5cd8a2869fc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:43.472039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:43.472039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:43.472039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:43.472039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:43.472039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:43.472039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:43.472039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:43.472039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:43.472039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:43.472039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:43.472039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:43.472039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:43.472039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:43.472039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:43.472039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:43.473083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:43.473083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:43.473083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:43.473083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:43.473083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:43.473083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:43.473083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:43.473083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:43.473083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:43.473083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:43.473083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:43.473083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:43.473083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:43.473083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:43.473083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:43.473083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:43.473083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:43.474041 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:43.474041 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:43.474041 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:43.478272 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 413/837] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:32:43 2022] GET /identity/v3/role_assignments?user.id=9daa7489bced467984d39e8b8ac49d13&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=e578ef831127463d9b48cc89fe20329c => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:43.478466 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 424/837] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:43.485407 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3202593-d47f-4574-a623-5cd8a2869fc5 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:43.548923 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:43.672993 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3202593-d47f-4574-a623-5cd8a2869fc5 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:43.715023 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e7a91-1e8a-42a0-8e5c-65fbeaa893a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:43.716051 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e7a91-1e8a-42a0-8e5c-65fbeaa893a4 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:43.716551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e7a91-1e8a-42a0-8e5c-65fbeaa893a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:43.717031 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e7a91-1e8a-42a0-8e5c-65fbeaa893a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:43.718153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e7a91-1e8a-42a0-8e5c-65fbeaa893a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:43.718698 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e7a91-1e8a-42a0-8e5c-65fbeaa893a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:43.720305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e7a91-1e8a-42a0-8e5c-65fbeaa893a4 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:43.735012 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 425/838] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:32:43.740720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc146f-01a0-494e-8194-0628e478f2a6 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:43.861015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:43.886960 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc146f-01a0-494e-8194-0628e478f2a6 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:43.914912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05356260-33e9-4571-8c40-7723e6e37007 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:43.915556 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05356260-33e9-4571-8c40-7723e6e37007 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:43.916059 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05356260-33e9-4571-8c40-7723e6e37007 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:43.917971 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 426/839] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:43.924076 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d50d83-1f93-4fc7-9c9a-57f704099b72 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:43.924696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d50d83-1f93-4fc7-9c9a-57f704099b72 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:43.925181 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d50d83-1f93-4fc7-9c9a-57f704099b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:43.946759 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad6ddf-9eb1-4bff-aca9-95ba8cba63e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:43.947480 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad6ddf-9eb1-4bff-aca9-95ba8cba63e7 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:43.947756 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad6ddf-9eb1-4bff-aca9-95ba8cba63e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:43.948021 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad6ddf-9eb1-4bff-aca9-95ba8cba63e7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:43.953339 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-d8d50d83-1f93-4fc7-9c9a-57f704099b72 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:43.954929 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-d8d50d83-1f93-4fc7-9c9a-57f704099b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:43.956967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad6ddf-9eb1-4bff-aca9-95ba8cba63e7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:43.959253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad6ddf-9eb1-4bff-aca9-95ba8cba63e7 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:43.973549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-88ad6ddf-9eb1-4bff-aca9-95ba8cba63e7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:32:43.973549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:43.973549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:43.973549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:43.973549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:43.973549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:43.973549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:43.973549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:43.973549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:43.973549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:43.973549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:43.973549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:43.973549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:43.973549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:43.973549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:43.973549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:43.974675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:43.974675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:43.974675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:43.974675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:43.974675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:43.974675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:43.974675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:43.974675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:43.974675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:43.974675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:43.974675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:43.974675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:43.974675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:43.974675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:43.974675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:43.974675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:43.974675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:43.974675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:43.975670 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:43.975670 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:43.975670 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:43.975670 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:43.975670 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:43.975670 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:43.975670 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:43.975670 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:43.975670 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:32:43.975670 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:43.975670 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 414/840] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:32:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:43.981379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad6ddf-9eb1-4bff-aca9-95ba8cba63e7 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:44.042679 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 427/841] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:32:44.059324 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d50d83-1f93-4fc7-9c9a-57f704099b72 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:44.075747 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:44.101216 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad6ddf-9eb1-4bff-aca9-95ba8cba63e7 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:44.143174 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68552f-6d33-4cf6-9adb-a438f27c7ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:44.144073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68552f-6d33-4cf6-9adb-a438f27c7ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:44.144499 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68552f-6d33-4cf6-9adb-a438f27c7ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:44.144930 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68552f-6d33-4cf6-9adb-a438f27c7ca4 None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:44.145928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68552f-6d33-4cf6-9adb-a438f27c7ca4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:44.146379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68552f-6d33-4cf6-9adb-a438f27c7ca4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:44.148139 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68552f-6d33-4cf6-9adb-a438f27c7ca4 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:44.167146 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:44.172436 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 415/842] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Aug 8 16:32:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:32:44.183948 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d50d83-1f93-4fc7-9c9a-57f704099b72 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:44.184174 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68552f-6d33-4cf6-9adb-a438f27c7ca4 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:44.218524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4925d62-7827-4fa8-a608-5f5de36d73ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:44.219331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4925d62-7827-4fa8-a608-5f5de36d73ed None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:44.220015 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4925d62-7827-4fa8-a608-5f5de36d73ed None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:44.220015 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4925d62-7827-4fa8-a608-5f5de36d73ed None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:44.220744 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4925d62-7827-4fa8-a608-5f5de36d73ed None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:44.222252 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4925d62-7827-4fa8-a608-5f5de36d73ed None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:44.233149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-c4925d62-7827-4fa8-a608-5f5de36d73ed None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 16:32:44.233149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:44.233149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:32:44.233149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:32:44.233149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:32:44.233149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:32:44.233149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:32:44.233149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:44.233149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 16:32:44.233149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:44.233149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:32:44.233149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:44.233149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:44.233149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:44.233149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:44.233149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:44.233149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:44.234415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:44.234415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:44.234415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:44.234415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:44.234415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:44.234415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:44.234415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:32:44.234415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:32:44.234415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:32:44.234415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:32:44.234415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:44.234415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:44.234415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:44.234415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:44.234415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:44.234415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:44.235367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:44.235367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:44.235367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:44.235367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:44.235367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:44.235367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:44.235367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:44.235367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:44.235367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:44.235367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:44.235367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:32:44.235367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:44.235367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 16:32:44.235367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:44.240335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 428/843] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Aug 8 16:32:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:44.241408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4925d62-7827-4fa8-a608-5f5de36d73ed None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:44.307429 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68552f-6d33-4cf6-9adb-a438f27c7ca4 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:44.331639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:44.348889 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb8f5f-fb53-400c-a1e3-74d7e4a7bcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:44.349883 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb8f5f-fb53-400c-a1e3-74d7e4a7bcbb None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:44.350343 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb8f5f-fb53-400c-a1e3-74d7e4a7bcbb None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:44.350828 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb8f5f-fb53-400c-a1e3-74d7e4a7bcbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:44.356287 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4925d62-7827-4fa8-a608-5f5de36d73ed None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:44.392564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a63d3-658c-4a85-9e2f-f2b4c9fa85a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:44.393252 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a63d3-658c-4a85-9e2f-f2b4c9fa85a0 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:44.393446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a63d3-658c-4a85-9e2f-f2b4c9fa85a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:44.393714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a63d3-658c-4a85-9e2f-f2b4c9fa85a0 None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:44.394354 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a63d3-658c-4a85-9e2f-f2b4c9fa85a0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:44.394682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a63d3-658c-4a85-9e2f-f2b4c9fa85a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:44.396164 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a63d3-658c-4a85-9e2f-f2b4c9fa85a0 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:44.406281 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 429/844] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Aug 8 16:32:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:44.407300 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a63d3-658c-4a85-9e2f-f2b4c9fa85a0 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:44.434129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:44.437814 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb8f5f-fb53-400c-a1e3-74d7e4a7bcbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:44.439729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb8f5f-fb53-400c-a1e3-74d7e4a7bcbb None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:44.515362 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a63d3-658c-4a85-9e2f-f2b4c9fa85a0 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:44.552083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635adeb-3bd6-4fc2-8156-178339e16636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:44.552669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635adeb-3bd6-4fc2-8156-178339e16636 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:44.552947 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635adeb-3bd6-4fc2-8156-178339e16636 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:44.553216 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635adeb-3bd6-4fc2-8156-178339e16636 None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:44.554100 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635adeb-3bd6-4fc2-8156-178339e16636 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:44.554212 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635adeb-3bd6-4fc2-8156-178339e16636 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:44.556472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635adeb-3bd6-4fc2-8156-178339e16636 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:44.563830 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 416/845] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:32:44 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:32:44.566770 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 430/846] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Aug 8 16:32:44 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:44.572832 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb8f5f-fb53-400c-a1e3-74d7e4a7bcbb None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:44.700595 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:44.707595 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb8f5f-fb53-400c-a1e3-74d7e4a7bcbb None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:44.750393 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a325cb-18b8-4543-bcf6-4acd7324d026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:44.751098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a325cb-18b8-4543-bcf6-4acd7324d026 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:44.751351 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a325cb-18b8-4543-bcf6-4acd7324d026 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:44.751623 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a325cb-18b8-4543-bcf6-4acd7324d026 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:44.765228 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a325cb-18b8-4543-bcf6-4acd7324d026 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:44.766851 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a325cb-18b8-4543-bcf6-4acd7324d026 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:44.787880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-75a325cb-18b8-4543-bcf6-4acd7324d026 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:44.787880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:44.787880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:44.787880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:44.787880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:44.787880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:44.787880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:44.787880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:44.787880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:44.787880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:44.787880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:44.787880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:44.787880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:44.787880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:44.787880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:44.787880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:44.788954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:44.788954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:44.788954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:44.788954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:44.788954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:44.788954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:44.788954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:44.788954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:44.788954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:44.788954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:44.788954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:44.788954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:44.788954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:44.788954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:44.788954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:44.788954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:44.788954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:44.789928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:44.789928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:44.789928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:44.790623 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 417/847] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:44.797916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635adeb-3bd6-4fc2-8156-178339e16636 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:44.814013 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:44.949992 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635adeb-3bd6-4fc2-8156-178339e16636 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:44.989604 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564995a-d512-4e9b-b2da-8b3eadebec55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:44.990159 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564995a-d512-4e9b-b2da-8b3eadebec55 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:44.990441 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564995a-d512-4e9b-b2da-8b3eadebec55 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:44.990768 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564995a-d512-4e9b-b2da-8b3eadebec55 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:44.991448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564995a-d512-4e9b-b2da-8b3eadebec55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:44.991748 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564995a-d512-4e9b-b2da-8b3eadebec55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:44.993774 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564995a-d512-4e9b-b2da-8b3eadebec55 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:45.008717 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 431/848] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:45.014117 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a325cb-18b8-4543-bcf6-4acd7324d026 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:45.133604 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a325cb-18b8-4543-bcf6-4acd7324d026 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:45.178958 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e7083-1eab-4353-96bd-19513da1e64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:45.179725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e7083-1eab-4353-96bd-19513da1e64a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:45.180642 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e7083-1eab-4353-96bd-19513da1e64a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:45.180914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e7083-1eab-4353-96bd-19513da1e64a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:45.196295 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e7083-1eab-4353-96bd-19513da1e64a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:45.197939 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e7083-1eab-4353-96bd-19513da1e64a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:45.216785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-7b1e7083-1eab-4353-96bd-19513da1e64a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:45.216785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:45.216785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:45.216785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:45.216785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:45.216785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:45.216785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:45.216785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:45.216785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:45.216785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:45.216785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:45.216785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:45.216785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:45.216785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:45.216785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:45.218039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:45.218039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:45.218039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:45.218039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:45.218039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:45.218039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:45.218039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:45.218039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:45.218039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:45.218039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:45.218039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:45.218039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:45.218039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:45.218039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:45.218039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:45.218039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:45.218039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:45.219479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:45.219479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:45.219479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:45.219479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:45.219479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:45.219479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:45.219479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:45.219479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:45.219479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 418/849] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:45.224237 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564995a-d512-4e9b-b2da-8b3eadebec55 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:45.275100 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb76d9-ee7e-4006-a356-5af3aa57142c None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:45.275840 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb76d9-ee7e-4006-a356-5af3aa57142c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:45.276319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb76d9-ee7e-4006-a356-5af3aa57142c None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:45.278504 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 419/850] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:32:45.285331 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5ef35-af12-420c-8132-9d8d7e8b97b2 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:45.285840 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5ef35-af12-420c-8132-9d8d7e8b97b2 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:45.286179 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5ef35-af12-420c-8132-9d8d7e8b97b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:45.312379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-c9b5ef35-af12-420c-8132-9d8d7e8b97b2 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:45.314846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-c9b5ef35-af12-420c-8132-9d8d7e8b97b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:45.317774 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:45.327983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:45.378276 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564995a-d512-4e9b-b2da-8b3eadebec55 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:45.435125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 420/851] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:32:45.441799 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec17d3e-7029-4e6c-9ae3-1d07fe3319fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:45.443358 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec17d3e-7029-4e6c-9ae3-1d07fe3319fb None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:45.443930 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec17d3e-7029-4e6c-9ae3-1d07fe3319fb None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:45.444593 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec17d3e-7029-4e6c-9ae3-1d07fe3319fb None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:45.446109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec17d3e-7029-4e6c-9ae3-1d07fe3319fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:45.446707 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec17d3e-7029-4e6c-9ae3-1d07fe3319fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:45.449750 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec17d3e-7029-4e6c-9ae3-1d07fe3319fb None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:45.458623 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5ef35-af12-420c-8132-9d8d7e8b97b2 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:45.479933 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 432/852] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:45.489776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec17d3e-7029-4e6c-9ae3-1d07fe3319fb None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:45.600086 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5ef35-af12-420c-8132-9d8d7e8b97b2 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:45.627943 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec17d3e-7029-4e6c-9ae3-1d07fe3319fb None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:45.654739 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0391ee-071a-447e-b5e6-f77e7808503b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:45.656201 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0391ee-071a-447e-b5e6-f77e7808503b None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:45.656475 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0391ee-071a-447e-b5e6-f77e7808503b None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:45.656744 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0391ee-071a-447e-b5e6-f77e7808503b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:45.663246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:45.674390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117107c-6b2b-4bb1-b2ef-ca8550f7b0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:45.675492 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117107c-6b2b-4bb1-b2ef-ca8550f7b0e8 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:45.675809 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117107c-6b2b-4bb1-b2ef-ca8550f7b0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:45.676181 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117107c-6b2b-4bb1-b2ef-ca8550f7b0e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:45.676595 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0391ee-071a-447e-b5e6-f77e7808503b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:32:45.677193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a117107c-6b2b-4bb1-b2ef-ca8550f7b0e8 None admin] RBAC: Adding query filter params (role.id=e578ef831127463d9b48cc89fe20329c, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=c606decb832444e38a194b98f8d15f8c) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:45.677602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a117107c-6b2b-4bb1-b2ef-ca8550f7b0e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:45.679297 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a117107c-6b2b-4bb1-b2ef-ca8550f7b0e8 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:45.682811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-8b0391ee-071a-447e-b5e6-f77e7808503b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:45.682811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:45.682811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:45.682811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:45.682811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:45.682811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:45.682811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:45.682811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:45.682811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:45.682811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:45.682811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:45.682811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:45.682811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:32:45.682811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:32:45.682811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:32:45.682811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:32:45.684064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:32:45.684064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:32:45.684064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:32:45.684064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:32:45.684064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:45.684064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:45.684064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:45.684064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:45.684064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:45.684064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:45.684064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:45.684064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:45.684064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:45.684064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:45.684064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:45.684064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:45.685104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:45.685104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:45.685104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:45.685104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:45.685104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:32:45.685104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:32:45.685104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:32:45.685104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:32:45.685104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:32:45.685104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:32:45.685104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:32:45.685104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:45.688749 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 421/853] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:32:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:32:45.694012 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 433/854] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:32:45 2022] GET /identity/v3/role_assignments?user.id=c606decb832444e38a194b98f8d15f8c&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=e578ef831127463d9b48cc89fe20329c => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:32:45.702682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0391ee-071a-447e-b5e6-f77e7808503b None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:45.705888 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:45.876967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0391ee-071a-447e-b5e6-f77e7808503b None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:45.944663 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9c9e8-5b2a-4d89-bccd-2062d04d47ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:45.946325 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9c9e8-5b2a-4d89-bccd-2062d04d47ff None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:45.946848 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9c9e8-5b2a-4d89-bccd-2062d04d47ff None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:45.947335 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9c9e8-5b2a-4d89-bccd-2062d04d47ff None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:45.948548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9c9e8-5b2a-4d89-bccd-2062d04d47ff None admin] RBAC: Adding query filter params (name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:45.949075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9c9e8-5b2a-4d89-bccd-2062d04d47ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:45.951867 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9c9e8-5b2a-4d89-bccd-2062d04d47ff None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:45.970257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 422/855] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Aug 8 16:32:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:45.980211 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117107c-6b2b-4bb1-b2ef-ca8550f7b0e8 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:46.109643 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117107c-6b2b-4bb1-b2ef-ca8550f7b0e8 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:46.155431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb46c5a-8a93-4549-a611-f79bb928b9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:46.156005 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46c5a-8a93-4549-a611-f79bb928b9c7 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:46.156277 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46c5a-8a93-4549-a611-f79bb928b9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:46.156542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46c5a-8a93-4549-a611-f79bb928b9c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:46.168834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1429ba-9eec-4ee6-adf0-b7aae01f3e4f None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:46.169321 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1429ba-9eec-4ee6-adf0-b7aae01f3e4f None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:46.169798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1429ba-9eec-4ee6-adf0-b7aae01f3e4f None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:46.170600 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb46c5a-8a93-4549-a611-f79bb928b9c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:46.172100 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb46c5a-8a93-4549-a611-f79bb928b9c7 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:46.172749 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 423/856] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:46.182478 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc513ba7-990a-40f9-b356-2bf8faf29305 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:46.182992 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc513ba7-990a-40f9-b356-2bf8faf29305 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:46.183452 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc513ba7-990a-40f9-b356-2bf8faf29305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:46.188690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-adb46c5a-8a93-4549-a611-f79bb928b9c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:46.188690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:46.188690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:46.188690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:46.188690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:46.188690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:46.188690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:46.188690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:46.188690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:46.188690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:46.188690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:46.188690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:46.188690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:46.188690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:46.188690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:46.188690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:46.189703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:46.189703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:46.189703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:46.189703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:46.189703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:46.189703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:46.189703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:46.189703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:46.189703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:46.189703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:46.189703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:46.189703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:46.189703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:46.189703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:46.189703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:46.189703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:46.189703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:46.191238 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:46.191238 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:46.191238 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:46.191238 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 434/857] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:46.198642 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb46c5a-8a93-4549-a611-f79bb928b9c7 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:46.218443 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-bc513ba7-990a-40f9-b356-2bf8faf29305 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:46.220863 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-bc513ba7-990a-40f9-b356-2bf8faf29305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:46.248348 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:46.313955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb46c5a-8a93-4549-a611-f79bb928b9c7 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:46.352553 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441e428-d463-4a6f-9b4a-330cf8cf678b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:46.353167 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:46.353334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441e428-d463-4a6f-9b4a-330cf8cf678b None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:46.353604 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441e428-d463-4a6f-9b4a-330cf8cf678b None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:46.353860 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441e428-d463-4a6f-9b4a-330cf8cf678b None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:46.354520 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441e428-d463-4a6f-9b4a-330cf8cf678b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:46.354852 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441e428-d463-4a6f-9b4a-330cf8cf678b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:46.356242 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441e428-d463-4a6f-9b4a-330cf8cf678b None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:46.374613 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 435/858] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:32:46.374900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 424/859] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:46 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:32:46.380403 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441e428-d463-4a6f-9b4a-330cf8cf678b None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:46.407798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc513ba7-990a-40f9-b356-2bf8faf29305 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:46.549503 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441e428-d463-4a6f-9b4a-330cf8cf678b None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:46.552517 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:46.606862 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc513ba7-990a-40f9-b356-2bf8faf29305 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:46.616511 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e02eb-2c3f-4b09-8178-4ff7ff0d4f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:46.618709 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e02eb-2c3f-4b09-8178-4ff7ff0d4f99 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:46.619197 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e02eb-2c3f-4b09-8178-4ff7ff0d4f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:46.619661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e02eb-2c3f-4b09-8178-4ff7ff0d4f99 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:46.633593 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e02eb-2c3f-4b09-8178-4ff7ff0d4f99 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:46.636321 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e02eb-2c3f-4b09-8178-4ff7ff0d4f99 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:46.658442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-653e02eb-2c3f-4b09-8178-4ff7ff0d4f99 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:32:46.658442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:46.658442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:46.658442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:46.658442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:46.658442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:46.658442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:46.658442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:46.658442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:46.658442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:46.658442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:46.658442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:46.658442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:46.658442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:46.658442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:46.658442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:46.660315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:46.660315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:46.660315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:46.660315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:46.660315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:46.660315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:46.660315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:46.660315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:46.660315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:46.660315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:46.660315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:46.660315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:46.660315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:46.660315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:46.660315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:46.660315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:46.660315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:46.660315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:46.662189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:46.662189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:46.662189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:46.662189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:46.662189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:46.662189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:46.662189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:46.662189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:46.662189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:32:46.662189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:46.667160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 436/860] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Aug 8 16:32:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:46.671548 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e02eb-2c3f-4b09-8178-4ff7ff0d4f99 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:46.684549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5601a3c-e0eb-473a-89c0-e53de3153c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:46.686077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5601a3c-e0eb-473a-89c0-e53de3153c12 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:46.686890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5601a3c-e0eb-473a-89c0-e53de3153c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:46.687623 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5601a3c-e0eb-473a-89c0-e53de3153c12 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:46.688998 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5601a3c-e0eb-473a-89c0-e53de3153c12 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:46.692295 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5601a3c-e0eb-473a-89c0-e53de3153c12 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:46.709762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-e5601a3c-e0eb-473a-89c0-e53de3153c12 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 16:32:46.709762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:46.709762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:32:46.709762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:32:46.709762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:32:46.709762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:32:46.709762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:32:46.709762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:46.709762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 16:32:46.709762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:46.709762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:32:46.709762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:46.709762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:46.709762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:46.709762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:46.709762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:46.711727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:46.711727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:46.711727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:46.711727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:46.711727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:46.711727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:46.711727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:46.711727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:32:46.711727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:32:46.711727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:32:46.711727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:32:46.711727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:46.711727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:46.711727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:46.711727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:46.711727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:46.711727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:46.713587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:46.713587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:46.713587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:46.713587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:46.713587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:46.713587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:46.713587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:46.713587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:46.713587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:46.713587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:46.713587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:32:46.713587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:46.713587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 16:32:46.713587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:46.722689 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 425/861] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Aug 8 16:32:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:32:46.730313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5601a3c-e0eb-473a-89c0-e53de3153c12 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:46.848219 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:46.866221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e02eb-2c3f-4b09-8178-4ff7ff0d4f99 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:46.932519 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c75f4-205b-44eb-bcc4-6c383ef7d22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:46.933500 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c75f4-205b-44eb-bcc4-6c383ef7d22c None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:46.934233 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c75f4-205b-44eb-bcc4-6c383ef7d22c None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:46.934818 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c75f4-205b-44eb-bcc4-6c383ef7d22c None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:46.936682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c75f4-205b-44eb-bcc4-6c383ef7d22c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:46.936682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c75f4-205b-44eb-bcc4-6c383ef7d22c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:46.939786 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c75f4-205b-44eb-bcc4-6c383ef7d22c None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:46.949338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5601a3c-e0eb-473a-89c0-e53de3153c12 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:46.979750 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 437/862] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Aug 8 16:32:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:46.985471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c75f4-205b-44eb-bcc4-6c383ef7d22c None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:47.020768 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9be36a-ac7a-42d6-b56f-09bb24d2178b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:47.022069 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9be36a-ac7a-42d6-b56f-09bb24d2178b None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:47.022761 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9be36a-ac7a-42d6-b56f-09bb24d2178b None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:47.023445 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9be36a-ac7a-42d6-b56f-09bb24d2178b None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:47.026211 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:47.027908 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9be36a-ac7a-42d6-b56f-09bb24d2178b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:47.028621 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9be36a-ac7a-42d6-b56f-09bb24d2178b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:47.031351 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9be36a-ac7a-42d6-b56f-09bb24d2178b None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:47.046975 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 426/863] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Aug 8 16:32:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:32:47.051454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9be36a-ac7a-42d6-b56f-09bb24d2178b None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:47.184508 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c75f4-205b-44eb-bcc4-6c383ef7d22c None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:47.224367 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9be36a-ac7a-42d6-b56f-09bb24d2178b None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:47.231124 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6708c-d395-497c-8c18-b7ec344d0d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:47.231695 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6708c-d395-497c-8c18-b7ec344d0d92 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:47.231973 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6708c-d395-497c-8c18-b7ec344d0d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:47.232240 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6708c-d395-497c-8c18-b7ec344d0d92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:47.253732 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:47.270548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52944d8-acfd-47d5-86f2-05293c9febd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:47.271484 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52944d8-acfd-47d5-86f2-05293c9febd7 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:47.271964 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52944d8-acfd-47d5-86f2-05293c9febd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:47.272409 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52944d8-acfd-47d5-86f2-05293c9febd7 None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:47.273394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52944d8-acfd-47d5-86f2-05293c9febd7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:47.273953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52944d8-acfd-47d5-86f2-05293c9febd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:47.276205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52944d8-acfd-47d5-86f2-05293c9febd7 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:47.288481 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 427/864] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Aug 8 16:32:47 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:47.288481 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3e0c0-2e8d-46bf-aab4-2aa5df5fa7d8 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:47.288481 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3e0c0-2e8d-46bf-aab4-2aa5df5fa7d8 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:47.288481 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3e0c0-2e8d-46bf-aab4-2aa5df5fa7d8 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:47.288481 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 428/865] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:47.294345 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334d974-4b37-4b65-b7a2-9dd7f9cc1f9c None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:47.294918 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334d974-4b37-4b65-b7a2-9dd7f9cc1f9c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:47.295377 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334d974-4b37-4b65-b7a2-9dd7f9cc1f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:47.325877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-6334d974-4b37-4b65-b7a2-9dd7f9cc1f9c None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:47.327873 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-6334d974-4b37-4b65-b7a2-9dd7f9cc1f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:47.363394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6708c-d395-497c-8c18-b7ec344d0d92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:47.366143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6708c-d395-497c-8c18-b7ec344d0d92 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:47.408570 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:47.495026 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 429/866] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:32:47.523990 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334d974-4b37-4b65-b7a2-9dd7f9cc1f9c None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:47.560443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 438/867] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:32:46 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:32:47.568026 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6708c-d395-497c-8c18-b7ec344d0d92 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:47.716188 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334d974-4b37-4b65-b7a2-9dd7f9cc1f9c None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:47.746129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6708c-d395-497c-8c18-b7ec344d0d92 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:47.762020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:47.770363 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:47.788415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9f517-7a1f-4ba1-a489-409fc1674e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:47.789213 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9f517-7a1f-4ba1-a489-409fc1674e8e None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:47.790113 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9f517-7a1f-4ba1-a489-409fc1674e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:47.790667 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9f517-7a1f-4ba1-a489-409fc1674e8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:47.803169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c9f517-7a1f-4ba1-a489-409fc1674e8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:47.805479 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c9f517-7a1f-4ba1-a489-409fc1674e8e None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:47.806094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577ec98-944f-4f1d-be68-dfeabf9141fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:47.809699 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577ec98-944f-4f1d-be68-dfeabf9141fa None admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:47.810668 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577ec98-944f-4f1d-be68-dfeabf9141fa None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:47.811382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577ec98-944f-4f1d-be68-dfeabf9141fa None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:47.812880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5577ec98-944f-4f1d-be68-dfeabf9141fa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:47.817249 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5577ec98-944f-4f1d-be68-dfeabf9141fa None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:47.824311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-f1c9f517-7a1f-4ba1-a489-409fc1674e8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:47.824311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:47.824311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:47.824311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:47.824311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:47.824311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:47.824311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:47.824311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:47.824311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:47.824311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:47.824311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:47.824311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:47.824311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:47.824311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:47.824311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:47.824311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:47.830647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:47.830647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:47.830647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:47.830647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:47.830647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:47.830647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:47.830647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:47.830647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:47.830647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:47.830647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:47.830647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:47.830647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:47.830647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:47.830647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:47.830647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:47.830647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:47.830647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:47.832453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:47.832453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:47.832453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:47.832453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 439/868] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:32:47.839836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9f517-7a1f-4ba1-a489-409fc1674e8e None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:47.935841 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 430/869] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Aug 8 16:32:47 2022] POST /identity/v3/services => generated 251 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:32:48.017596 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9f517-7a1f-4ba1-a489-409fc1674e8e None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:48.088238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73592a3e-5ed9-42b7-bba8-655f6e376df0 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:48.088777 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73592a3e-5ed9-42b7-bba8-655f6e376df0 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:48.088777 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73592a3e-5ed9-42b7-bba8-655f6e376df0 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:48.090255 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 431/870] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:48.100107 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20283823-8e04-471f-9ce1-e4a49e50ce75 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:48.100365 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20283823-8e04-471f-9ce1-e4a49e50ce75 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:48.100722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20283823-8e04-471f-9ce1-e4a49e50ce75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:48.105403 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7accfcf7-c974-4a12-adcf-ac34e19f84f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:48.106220 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accfcf7-c974-4a12-adcf-ac34e19f84f0 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:48.110584 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accfcf7-c974-4a12-adcf-ac34e19f84f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:48.110584 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accfcf7-c974-4a12-adcf-ac34e19f84f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:48.110584 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7accfcf7-c974-4a12-adcf-ac34e19f84f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:48.110584 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7accfcf7-c974-4a12-adcf-ac34e19f84f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:48.112288 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7accfcf7-c974-4a12-adcf-ac34e19f84f0 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:48.126594 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-20283823-8e04-471f-9ce1-e4a49e50ce75 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:48.129140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-20283823-8e04-471f-9ce1-e4a49e50ce75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:48.134012 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 440/871] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:48.140554 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7accfcf7-c974-4a12-adcf-ac34e19f84f0 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:48.197928 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:48.252223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 432/872] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:48 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:32:48.289026 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20283823-8e04-471f-9ce1-e4a49e50ce75 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:48.292294 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7accfcf7-c974-4a12-adcf-ac34e19f84f0 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:48.348924 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250069fd-0fa7-402e-ad3c-f624658a23ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:48.350099 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250069fd-0fa7-402e-ad3c-f624658a23ce None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:48.350382 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250069fd-0fa7-402e-ad3c-f624658a23ce None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:48.350680 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250069fd-0fa7-402e-ad3c-f624658a23ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:48.360770 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250069fd-0fa7-402e-ad3c-f624658a23ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:48.362225 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250069fd-0fa7-402e-ad3c-f624658a23ce None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:48.380648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-250069fd-0fa7-402e-ad3c-f624658a23ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:48.380648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:48.380648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:48.380648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:48.380648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:48.380648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:48.380648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:48.380648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:48.380648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:48.380648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:48.380648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:48.380648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:48.380648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:48.380648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:48.380648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:48.382390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:48.382390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:48.382390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:48.382390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:48.382390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:48.382390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:48.382390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:48.382390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:48.382390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:48.382390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:48.382390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:48.382390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:48.382390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:48.382390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:48.382390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:48.382390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:48.382390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:48.384267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:48.384267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:48.384267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:48.384267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:48.384267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:48.384267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:48.384267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:48.384267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:48.384267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 441/873] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:32:48.394951 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250069fd-0fa7-402e-ad3c-f624658a23ce None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:48.478213 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20283823-8e04-471f-9ce1-e4a49e50ce75 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:48.492786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:48.538446 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1e4df-a8be-4419-a487-863da8bd2870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:48.540424 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1e4df-a8be-4419-a487-863da8bd2870 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:48.542152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1e4df-a8be-4419-a487-863da8bd2870 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:48.543020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1e4df-a8be-4419-a487-863da8bd2870 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:48.544079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1e4df-a8be-4419-a487-863da8bd2870 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:48.546698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1e4df-a8be-4419-a487-863da8bd2870 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:48.563241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-c4c1e4df-a8be-4419-a487-863da8bd2870 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:32:48.563241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:48.563241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:32:48.563241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:32:48.563241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:32:48.563241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:32:48.563241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:32:48.563241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:48.563241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:32:48.563241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:48.563241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:32:48.563241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:48.563241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:48.563241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:48.563241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:48.563241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:48.563241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:48.565631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:48.565631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:48.565631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:48.565631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:48.565631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:48.565631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:48.565631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:32:48.565631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:32:48.565631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:32:48.565631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:32:48.565631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:48.565631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:48.565631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:48.565631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:48.565631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:48.565631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:48.567683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:48.567683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:48.567683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:48.567683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:48.567683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:48.567683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:48.567683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:48.567683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:48.567683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:48.567683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:48.567683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:32:48.567683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:48.567683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:32:48.567683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:48.574961 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 433/874] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Aug 8 16:32:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:48.580752 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1e4df-a8be-4419-a487-863da8bd2870 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:48.606882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:48.627482 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250069fd-0fa7-402e-ad3c-f624658a23ce None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:48.687563 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153f7ef-629b-4814-886f-7c74b25ca20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:48.688752 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153f7ef-629b-4814-886f-7c74b25ca20a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:48.689317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153f7ef-629b-4814-886f-7c74b25ca20a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:48.689877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153f7ef-629b-4814-886f-7c74b25ca20a None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:48.691207 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153f7ef-629b-4814-886f-7c74b25ca20a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:48.691826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153f7ef-629b-4814-886f-7c74b25ca20a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:48.694505 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153f7ef-629b-4814-886f-7c74b25ca20a None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:48.723073 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 442/875] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:48.728709 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153f7ef-629b-4814-886f-7c74b25ca20a None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:48.782215 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1e4df-a8be-4419-a487-863da8bd2870 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:48.845118 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf90f14-22d9-4e58-804d-7b091215ec16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:48.846359 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf90f14-22d9-4e58-804d-7b091215ec16 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:48.847038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf90f14-22d9-4e58-804d-7b091215ec16 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:48.847764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf90f14-22d9-4e58-804d-7b091215ec16 None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:48.849037 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf90f14-22d9-4e58-804d-7b091215ec16 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:48.849806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf90f14-22d9-4e58-804d-7b091215ec16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:48.853983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf90f14-22d9-4e58-804d-7b091215ec16 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:48.863398 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 434/876] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Aug 8 16:32:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:48.871255 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf90f14-22d9-4e58-804d-7b091215ec16 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:48.916058 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153f7ef-629b-4814-886f-7c74b25ca20a None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:48.954839 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:48.998995 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca43c28e-887a-4d84-8eea-091c1fb1694f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:49.000421 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43c28e-887a-4d84-8eea-091c1fb1694f None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:49.001165 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43c28e-887a-4d84-8eea-091c1fb1694f None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:49.001875 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43c28e-887a-4d84-8eea-091c1fb1694f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:49.003583 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca43c28e-887a-4d84-8eea-091c1fb1694f None admin] RBAC: Adding query filter params (role.id=7764573472d34ecc8bedf7a9c7935170, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=9646662b0b3048f79ad5d2b9602f10a6) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:49.004419 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca43c28e-887a-4d84-8eea-091c1fb1694f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:49.007272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca43c28e-887a-4d84-8eea-091c1fb1694f None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:49.023102 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 443/877] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:32:48 2022] GET /identity/v3/role_assignments?user.id=9646662b0b3048f79ad5d2b9602f10a6&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=7764573472d34ecc8bedf7a9c7935170 => generated 608 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:49.084734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf90f14-22d9-4e58-804d-7b091215ec16 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:49.148716 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a495651a-3813-4253-a750-7c98cf262712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:49.149748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495651a-3813-4253-a750-7c98cf262712 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:49.150109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495651a-3813-4253-a750-7c98cf262712 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:49.150541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495651a-3813-4253-a750-7c98cf262712 None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:49.151693 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495651a-3813-4253-a750-7c98cf262712 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:49.152188 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495651a-3813-4253-a750-7c98cf262712 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:49.154516 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495651a-3813-4253-a750-7c98cf262712 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:49.163220 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 435/878] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Aug 8 16:32:48 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:32:49.906743 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a558f-ce26-4157-b445-0a0af419a376 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:49.907714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a558f-ce26-4157-b445-0a0af419a376 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:49.907714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a558f-ce26-4157-b445-0a0af419a376 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:49.908927 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 444/879] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:49.915011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28435c85-947b-4fdf-bb0d-03793be34ae4 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:49.915387 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28435c85-947b-4fdf-bb0d-03793be34ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:49.915628 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28435c85-947b-4fdf-bb0d-03793be34ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:49.942831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-28435c85-947b-4fdf-bb0d-03793be34ae4 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:49.944438 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-28435c85-947b-4fdf-bb0d-03793be34ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:50.023114 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 436/880] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:49 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:32:50.046516 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a558f-ce26-4157-b445-0a0af419a376 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:50.062880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:50.162686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a558f-ce26-4157-b445-0a0af419a376 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:50.204490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f7e25-a18e-45a5-abd3-d4c91511f0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:50.205682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f7e25-a18e-45a5-abd3-d4c91511f0ab None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:50.206171 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f7e25-a18e-45a5-abd3-d4c91511f0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:50.206653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f7e25-a18e-45a5-abd3-d4c91511f0ab None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:50.207501 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f7e25-a18e-45a5-abd3-d4c91511f0ab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:50.209356 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f7e25-a18e-45a5-abd3-d4c91511f0ab None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:50.294664 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 445/881] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Aug 8 16:32:50 2022] POST /identity/v3/services => generated 269 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:32:50.467385 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b531b-2073-471a-9338-c1438e606bbe None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:50.467902 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b531b-2073-471a-9338-c1438e606bbe None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:50.467902 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b531b-2073-471a-9338-c1438e606bbe None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:50.469499 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 437/882] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:50.475756 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ff42e-f3fb-4ee0-a121-109fc5cd5776 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:50.476075 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ff42e-f3fb-4ee0-a121-109fc5cd5776 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:50.476378 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ff42e-f3fb-4ee0-a121-109fc5cd5776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:50.508632 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-0c0ff42e-f3fb-4ee0-a121-109fc5cd5776 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:50.510670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-0c0ff42e-f3fb-4ee0-a121-109fc5cd5776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:50.615040 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:50.625809 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 446/883] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:50 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:32:50.651646 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b531b-2073-471a-9338-c1438e606bbe None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:50.689695 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:50.831005 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b531b-2073-471a-9338-c1438e606bbe None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:50.904968 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31223226-8c44-49a5-8344-f6323ebf11f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:50.906359 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31223226-8c44-49a5-8344-f6323ebf11f3 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:50.907121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31223226-8c44-49a5-8344-f6323ebf11f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:50.907864 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31223226-8c44-49a5-8344-f6323ebf11f3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:50.909171 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31223226-8c44-49a5-8344-f6323ebf11f3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:50.912131 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31223226-8c44-49a5-8344-f6323ebf11f3 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:50.931567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-31223226-8c44-49a5-8344-f6323ebf11f3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 16:32:50.931567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:50.931567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:32:50.931567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:32:50.931567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:32:50.931567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:32:50.931567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:32:50.931567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:50.931567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 16:32:50.931567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:50.931567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:32:50.931567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:50.931567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:50.931567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:50.931567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:50.931567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:50.931567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:50.932508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:50.932508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:50.932508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:50.932508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:50.932508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:50.932508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:50.932508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:32:50.932508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:32:50.932508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:32:50.932508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:32:50.932508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:50.932508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:50.932508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:50.932508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:50.932508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:50.932508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:50.933364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:50.933364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:50.933364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:50.933364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:50.933364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:50.933364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:50.933364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:50.933364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:50.933364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:50.933364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:50.933364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:32:50.933364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:50.933364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 16:32:50.933364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:50.934122 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 438/884] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Aug 8 16:32:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:50.943084 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ff42e-f3fb-4ee0-a121-109fc5cd5776 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:51.127875 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ff42e-f3fb-4ee0-a121-109fc5cd5776 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:51.194293 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8b8e3-165a-4f10-8163-dbfae08eddef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:51.196338 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8b8e3-165a-4f10-8163-dbfae08eddef None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:51.196979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8b8e3-165a-4f10-8163-dbfae08eddef None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:51.197476 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8b8e3-165a-4f10-8163-dbfae08eddef None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:51.198658 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8b8e3-165a-4f10-8163-dbfae08eddef None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:51.199203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8b8e3-165a-4f10-8163-dbfae08eddef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:51.201785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8b8e3-165a-4f10-8163-dbfae08eddef None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:51.212125 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 447/885] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Aug 8 16:32:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:51.219737 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31223226-8c44-49a5-8344-f6323ebf11f3 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:51.357602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b735c7-db1b-49ed-bb39-4e97f6c07de3 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:51.358992 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:51.359394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b735c7-db1b-49ed-bb39-4e97f6c07de3 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:51.361003 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b735c7-db1b-49ed-bb39-4e97f6c07de3 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:51.365375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 448/886] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:51 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:51.372578 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31223226-8c44-49a5-8344-f6323ebf11f3 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:51.376541 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2542761-ea08-4b74-8b53-46c9d32b17fb None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:51.376541 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2542761-ea08-4b74-8b53-46c9d32b17fb None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:51.376541 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2542761-ea08-4b74-8b53-46c9d32b17fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:51.393629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-f2542761-ea08-4b74-8b53-46c9d32b17fb None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:51.394764 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f2542761-ea08-4b74-8b53-46c9d32b17fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:51.427523 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672b173-7a0d-46f8-8a49-92138924a207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:51.428862 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672b173-7a0d-46f8-8a49-92138924a207 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:51.428862 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672b173-7a0d-46f8-8a49-92138924a207 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:51.428862 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672b173-7a0d-46f8-8a49-92138924a207 None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:51.429469 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672b173-7a0d-46f8-8a49-92138924a207 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:51.429728 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672b173-7a0d-46f8-8a49-92138924a207 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:51.432084 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672b173-7a0d-46f8-8a49-92138924a207 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:51.438385 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 439/887] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Aug 8 16:32:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:51.456077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672b173-7a0d-46f8-8a49-92138924a207 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:51.538327 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:51.564691 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 449/888] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:51 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:32:51.589957 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2542761-ea08-4b74-8b53-46c9d32b17fb None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:51.603605 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672b173-7a0d-46f8-8a49-92138924a207 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:51.645865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8531d1-8c81-442d-bb28-31605cba063f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:51.646526 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8531d1-8c81-442d-bb28-31605cba063f None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:51.646937 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8531d1-8c81-442d-bb28-31605cba063f None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:51.647291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8531d1-8c81-442d-bb28-31605cba063f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:51.649169 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8531d1-8c81-442d-bb28-31605cba063f None admin] RBAC: Adding query filter params (interface=public, service_id=e03df074784a41f593fa15c560394ed2, region_id=RegionOne) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:51.649169 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8531d1-8c81-442d-bb28-31605cba063f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:51.651054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8531d1-8c81-442d-bb28-31605cba063f None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:51.657629 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 440/889] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Aug 8 16:32:51 2022] GET /identity/v3/endpoints?service_id=e03df074784a41f593fa15c560394ed2&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:51.667729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8531d1-8c81-442d-bb28-31605cba063f None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:51.693274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:51.767803 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2542761-ea08-4b74-8b53-46c9d32b17fb None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:51.836110 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640c1e0-4b5c-4730-9359-75809fe154df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:51.837873 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640c1e0-4b5c-4730-9359-75809fe154df None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:51.838042 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8531d1-8c81-442d-bb28-31605cba063f None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:51.838315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640c1e0-4b5c-4730-9359-75809fe154df None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:51.838752 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640c1e0-4b5c-4730-9359-75809fe154df None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:51.858616 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7640c1e0-4b5c-4730-9359-75809fe154df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:32:51.863918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-7640c1e0-4b5c-4730-9359-75809fe154df None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:32:51.863918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:51.863918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:51.863918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:51.863918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:51.863918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:51.863918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:51.863918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:51.863918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:51.863918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:51.863918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:51.863918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:51.863918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:32:51.863918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:32:51.863918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:32:51.863918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:32:51.865987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:32:51.865987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:32:51.865987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:32:51.865987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:32:51.865987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:51.865987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:51.865987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:51.865987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:51.865987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:51.865987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:51.865987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:51.865987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:51.865987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:51.865987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:51.865987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:51.865987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:51.868518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:51.868518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:51.868518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:51.868518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:51.868518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:32:51.868518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:32:51.868518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:32:51.868518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:32:51.868518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:32:51.868518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:32:51.868518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:32:51.868518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:51.870801 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 450/890] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Aug 8 16:32:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:51.874961 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7212a-98b5-469a-bc8d-c5b099bf4710 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:51.875451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7212a-98b5-469a-bc8d-c5b099bf4710 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:51.875929 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7212a-98b5-469a-bc8d-c5b099bf4710 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:51.878481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 451/891] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:51.879926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7212a-98b5-469a-bc8d-c5b099bf4710 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:51.916728 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09765dde-96b7-4072-8c1e-9c2ddc6d2fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:51.917567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09765dde-96b7-4072-8c1e-9c2ddc6d2fbe None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:51.918044 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09765dde-96b7-4072-8c1e-9c2ddc6d2fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:51.918516 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09765dde-96b7-4072-8c1e-9c2ddc6d2fbe None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:51.921985 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09765dde-96b7-4072-8c1e-9c2ddc6d2fbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:51.922378 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09765dde-96b7-4072-8c1e-9c2ddc6d2fbe None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:51.935074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 441/892] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Aug 8 16:32:51 2022] GET /identity/v3/services => generated 1268 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:32:51.937386 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817a940-f8bf-4328-9e43-e91101d587c3 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:51.937874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817a940-f8bf-4328-9e43-e91101d587c3 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:51.938341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817a940-f8bf-4328-9e43-e91101d587c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:51.979665 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-7817a940-f8bf-4328-9e43-e91101d587c3 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:51.981657 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:51.985770 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-7817a940-f8bf-4328-9e43-e91101d587c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:52.085580 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7212a-98b5-469a-bc8d-c5b099bf4710 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:52.154520 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 442/893] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:51 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:32:52.161654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed60549-bd22-4a58-82b0-b319cc7099a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:52.162773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed60549-bd22-4a58-82b0-b319cc7099a6 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:52.163262 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed60549-bd22-4a58-82b0-b319cc7099a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:52.163736 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed60549-bd22-4a58-82b0-b319cc7099a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:52.165465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed60549-bd22-4a58-82b0-b319cc7099a6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:52.165989 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed60549-bd22-4a58-82b0-b319cc7099a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:52.168682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed60549-bd22-4a58-82b0-b319cc7099a6 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:52.183134 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817a940-f8bf-4328-9e43-e91101d587c3 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:52.188475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 452/894] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Aug 8 16:32:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:52.193138 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed60549-bd22-4a58-82b0-b319cc7099a6 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:52.218688 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:52.386729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed60549-bd22-4a58-82b0-b319cc7099a6 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:52.388826 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817a940-f8bf-4328-9e43-e91101d587c3 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:52.416359 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:52.453862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c3c64-19d8-40f2-b2ca-52513d8aac21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:52.455711 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c3c64-19d8-40f2-b2ca-52513d8aac21 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:52.456855 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14e2e4-f637-41fe-9ea2-68d6683dd63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:52.457058 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c3c64-19d8-40f2-b2ca-52513d8aac21 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:52.457615 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c3c64-19d8-40f2-b2ca-52513d8aac21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:52.459093 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14e2e4-f637-41fe-9ea2-68d6683dd63f None admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:52.460186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14e2e4-f637-41fe-9ea2-68d6683dd63f None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:52.460844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14e2e4-f637-41fe-9ea2-68d6683dd63f None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:52.463126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad14e2e4-f637-41fe-9ea2-68d6683dd63f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:52.466028 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad14e2e4-f637-41fe-9ea2-68d6683dd63f None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:52.480381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c3c64-19d8-40f2-b2ca-52513d8aac21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:52.483317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c3c64-19d8-40f2-b2ca-52513d8aac21 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:52.513562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-f21c3c64-19d8-40f2-b2ca-52513d8aac21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:52.513562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:52.513562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:52.513562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:52.513562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:52.513562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:52.513562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:52.513562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:52.513562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:52.513562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:52.513562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:52.513562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:52.513562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:52.513562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:52.513562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:52.513562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:52.515524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:52.515524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:52.515524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:52.515524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:52.515524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:52.515524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:52.515524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:52.515524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:52.515524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:52.515524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:52.515524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:52.515524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:52.515524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:52.515524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:52.515524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:52.515524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:52.515524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:52.522704 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:52.522704 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:52.522704 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:52.522704 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 453/895] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:52.525829 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c3c64-19d8-40f2-b2ca-52513d8aac21 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:52.565251 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 443/896] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Aug 8 16:32:52 2022] POST /identity/v3/services => generated 241 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:32:52.567987 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b10a6-31a1-4805-8e68-d3d18b6ec957 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:52.568825 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b10a6-31a1-4805-8e68-d3d18b6ec957 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:52.569326 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b10a6-31a1-4805-8e68-d3d18b6ec957 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:52.572803 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 444/897] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:52.588276 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62631f-8b9b-4dc9-b605-be7a72aacbd2 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:52.589210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62631f-8b9b-4dc9-b605-be7a72aacbd2 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:52.589773 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62631f-8b9b-4dc9-b605-be7a72aacbd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:52.624152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-5d62631f-8b9b-4dc9-b605-be7a72aacbd2 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:52.627045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-5d62631f-8b9b-4dc9-b605-be7a72aacbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:52.717391 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c3c64-19d8-40f2-b2ca-52513d8aac21 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:52.765287 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f361616-7879-4265-89fd-1a4a1e997784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:52.765899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f361616-7879-4265-89fd-1a4a1e997784 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:52.766169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f361616-7879-4265-89fd-1a4a1e997784 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:52.766435 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f361616-7879-4265-89fd-1a4a1e997784 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:52.767104 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f361616-7879-4265-89fd-1a4a1e997784 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:52.767398 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f361616-7879-4265-89fd-1a4a1e997784 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:52.768764 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f361616-7879-4265-89fd-1a4a1e997784 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:52.779038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 445/898] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:32:52.782169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 454/899] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:52.792084 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62631f-8b9b-4dc9-b605-be7a72aacbd2 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:52.798492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:52.810818 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f361616-7879-4265-89fd-1a4a1e997784 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:52.840448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:52.940252 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f361616-7879-4265-89fd-1a4a1e997784 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:52.955741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62631f-8b9b-4dc9-b605-be7a72aacbd2 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:52.981127 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009b3ca-5503-400d-9adc-cfa3b84a8bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:52.982084 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009b3ca-5503-400d-9adc-cfa3b84a8bdc None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:52.983005 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009b3ca-5503-400d-9adc-cfa3b84a8bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:52.983550 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009b3ca-5503-400d-9adc-cfa3b84a8bdc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:52.984368 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009b3ca-5503-400d-9adc-cfa3b84a8bdc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:52.985994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009b3ca-5503-400d-9adc-cfa3b84a8bdc None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:52.996883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-4009b3ca-5503-400d-9adc-cfa3b84a8bdc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 16:32:52.996883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:52.996883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:32:52.996883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:32:52.996883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:32:52.996883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:32:52.996883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:32:52.996883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:52.996883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 16:32:52.996883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:52.996883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:32:52.996883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:52.996883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:52.996883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:52.996883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:52.996883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:52.997937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:52.997937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:52.997937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:52.997937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:52.997937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:52.997937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:52.997937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:52.997937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:32:52.997937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:32:52.997937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:32:52.997937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:32:52.997937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:52.997937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:52.997937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:52.997937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:52.997937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:52.997937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:52.998916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:52.998916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:52.998916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:52.998916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:52.998916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:52.998916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:52.998916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:52.998916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:52.998916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:52.998916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:52.998916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:32:52.998916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:52.998916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 16:32:52.998916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:53.002670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 455/900] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Aug 8 16:32:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:32:53.006367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009b3ca-5503-400d-9adc-cfa3b84a8bdc None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:53.009239 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f23ae5-a60b-41b2-a6c8-e08fb776e7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:53.009973 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f23ae5-a60b-41b2-a6c8-e08fb776e7d9 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:53.010247 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f23ae5-a60b-41b2-a6c8-e08fb776e7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:53.010515 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f23ae5-a60b-41b2-a6c8-e08fb776e7d9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:53.021123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f23ae5-a60b-41b2-a6c8-e08fb776e7d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:53.023366 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f23ae5-a60b-41b2-a6c8-e08fb776e7d9 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:53.041822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-a6f23ae5-a60b-41b2-a6c8-e08fb776e7d9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:32:53.041822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:53.041822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:53.041822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:53.041822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:53.041822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:53.041822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:53.041822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:53.041822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:53.041822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:53.041822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:53.041822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:53.041822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:53.041822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:53.041822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:53.041822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:53.042928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:53.042928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:53.042928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:53.042928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:53.042928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:53.042928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:53.042928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:53.042928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:53.042928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:53.042928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:53.042928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:53.042928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:53.042928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:53.042928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:53.042928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:53.042928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:53.042928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:53.042928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:53.043912 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:53.043912 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:53.043912 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:53.043912 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:53.043912 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:53.043912 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:53.043912 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:53.043912 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:53.043912 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:32:53.043912 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:53.043912 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 446/901] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Aug 8 16:32:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:53.050947 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f23ae5-a60b-41b2-a6c8-e08fb776e7d9 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:53.115388 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:53.133538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009b3ca-5503-400d-9adc-cfa3b84a8bdc None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:53.140307 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:53.174283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3aca0e-9da0-4677-9553-3748480d8fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:53.175293 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3aca0e-9da0-4677-9553-3748480d8fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:53.175793 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3aca0e-9da0-4677-9553-3748480d8fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:53.176325 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3aca0e-9da0-4677-9553-3748480d8fe5 None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:53.177197 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3aca0e-9da0-4677-9553-3748480d8fe5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:53.177763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3aca0e-9da0-4677-9553-3748480d8fe5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:53.179360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3aca0e-9da0-4677-9553-3748480d8fe5 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:53.185528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 456/902] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Aug 8 16:32:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:53.192305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3aca0e-9da0-4677-9553-3748480d8fe5 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:53.194421 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f23ae5-a60b-41b2-a6c8-e08fb776e7d9 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:53.234203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf6044-134f-4e94-b8ef-455d226f3772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:53.235250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf6044-134f-4e94-b8ef-455d226f3772 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:53.235831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf6044-134f-4e94-b8ef-455d226f3772 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:53.236301 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf6044-134f-4e94-b8ef-455d226f3772 None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:53.237256 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf6044-134f-4e94-b8ef-455d226f3772 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:53.237766 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf6044-134f-4e94-b8ef-455d226f3772 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:53.239538 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf6044-134f-4e94-b8ef-455d226f3772 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:53.264643 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 447/903] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Aug 8 16:32:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:53.265942 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf6044-134f-4e94-b8ef-455d226f3772 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:53.323375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3aca0e-9da0-4677-9553-3748480d8fe5 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:53.367011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38c99a-0643-4462-aa58-e4089ee40c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:53.367752 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38c99a-0643-4462-aa58-e4089ee40c49 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:53.368212 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38c99a-0643-4462-aa58-e4089ee40c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:53.368636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38c99a-0643-4462-aa58-e4089ee40c49 None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:53.369345 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d38c99a-0643-4462-aa58-e4089ee40c49 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:53.369676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d38c99a-0643-4462-aa58-e4089ee40c49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:53.371766 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d38c99a-0643-4462-aa58-e4089ee40c49 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:53.382767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 457/904] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Aug 8 16:32:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:53.388399 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38c99a-0643-4462-aa58-e4089ee40c49 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:53.405170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf6044-134f-4e94-b8ef-455d226f3772 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:53.413859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:53.432007 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:53.443942 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b392b85-ce5c-46e6-9bce-a3fb8f1c6e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:53.444436 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b392b85-ce5c-46e6-9bce-a3fb8f1c6e77 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:53.445026 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b392b85-ce5c-46e6-9bce-a3fb8f1c6e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:53.445399 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b392b85-ce5c-46e6-9bce-a3fb8f1c6e77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:53.524975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38c99a-0643-4462-aa58-e4089ee40c49 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:53.544434 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b392b85-ce5c-46e6-9bce-a3fb8f1c6e77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:53.546244 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b392b85-ce5c-46e6-9bce-a3fb8f1c6e77 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:53.563895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b4e12-0544-4e14-8454-d5c57a99b05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:53.564609 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b4e12-0544-4e14-8454-d5c57a99b05c None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:53.564955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b4e12-0544-4e14-8454-d5c57a99b05c None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:53.565292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b4e12-0544-4e14-8454-d5c57a99b05c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:53.566078 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b4e12-0544-4e14-8454-d5c57a99b05c None admin] RBAC: Adding query filter params (interface=public, service_id=ce8a90a556d640efa3bb02cc0f276772, region_id=RegionOne) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:53.566553 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b4e12-0544-4e14-8454-d5c57a99b05c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:53.568133 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b4e12-0544-4e14-8454-d5c57a99b05c None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:53.574611 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 458/905] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Aug 8 16:32:53 2022] GET /identity/v3/endpoints?service_id=ce8a90a556d640efa3bb02cc0f276772&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:53.579583 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b4e12-0544-4e14-8454-d5c57a99b05c None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:53.688352 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 448/906] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:32:53 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:32:53.689850 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:53.690303 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b392b85-ce5c-46e6-9bce-a3fb8f1c6e77 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:53.716806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:53.720554 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b4e12-0544-4e14-8454-d5c57a99b05c None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:53.772062 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e175d4-6d57-4427-885b-14fe4b898062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:53.772783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e175d4-6d57-4427-885b-14fe4b898062 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:53.774396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e175d4-6d57-4427-885b-14fe4b898062 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:53.774901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e175d4-6d57-4427-885b-14fe4b898062 None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:53.775787 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e175d4-6d57-4427-885b-14fe4b898062 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:53.777215 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e175d4-6d57-4427-885b-14fe4b898062 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:53.786710 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 459/907] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Aug 8 16:32:53 2022] GET /identity/v3/services => generated 1497 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:32:53.855090 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b392b85-ce5c-46e6-9bce-a3fb8f1c6e77 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:53.928981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60d70e-4173-4255-a85f-bffd17f1e257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:53.930143 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60d70e-4173-4255-a85f-bffd17f1e257 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:53.930626 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60d70e-4173-4255-a85f-bffd17f1e257 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:53.934609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60d70e-4173-4255-a85f-bffd17f1e257 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:53.954796 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60d70e-4173-4255-a85f-bffd17f1e257 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:53.957619 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60d70e-4173-4255-a85f-bffd17f1e257 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:53.988893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-2c60d70e-4173-4255-a85f-bffd17f1e257 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:53.988893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:53.988893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:53.988893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:53.988893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:53.988893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:53.988893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:53.988893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:53.988893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:53.988893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:53.988893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:53.988893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:53.988893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:53.988893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:53.988893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:53.988893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:53.989917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:53.989917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:53.989917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:53.989917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:53.989917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:53.989917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:53.989917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:53.989917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:53.989917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:53.989917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:53.989917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:53.989917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:53.989917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:53.989917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:53.989917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:53.989917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:53.989917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:53.990864 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:53.990864 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:53.990864 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:53.991472 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 449/908] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:53.998083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e175d4-6d57-4427-885b-14fe4b898062 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:54.128478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e175d4-6d57-4427-885b-14fe4b898062 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:54.166700 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df479c8-54dd-4c01-b1db-cd6c7f9288ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:54.167323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df479c8-54dd-4c01-b1db-cd6c7f9288ad None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:54.167654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df479c8-54dd-4c01-b1db-cd6c7f9288ad None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:54.167899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df479c8-54dd-4c01-b1db-cd6c7f9288ad None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:54.168566 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df479c8-54dd-4c01-b1db-cd6c7f9288ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:54.168862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df479c8-54dd-4c01-b1db-cd6c7f9288ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:54.170244 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df479c8-54dd-4c01-b1db-cd6c7f9288ad None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:54.184549 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 460/909] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:54.194144 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60d70e-4173-4255-a85f-bffd17f1e257 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:54.340616 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:54.390729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60d70e-4173-4255-a85f-bffd17f1e257 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:54.443921 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df99317-9ea5-45b6-ae0d-88faff1fa468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:54.444809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df99317-9ea5-45b6-ae0d-88faff1fa468 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:54.451008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df99317-9ea5-45b6-ae0d-88faff1fa468 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:54.451008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df99317-9ea5-45b6-ae0d-88faff1fa468 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:54.454800 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df99317-9ea5-45b6-ae0d-88faff1fa468 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:54.456560 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df99317-9ea5-45b6-ae0d-88faff1fa468 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:54.471088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-5df99317-9ea5-45b6-ae0d-88faff1fa468 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:54.471088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:54.471088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:54.471088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:54.471088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:54.471088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:54.471088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:54.471088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:54.471088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:54.471088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:54.471088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:54.471088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:54.471088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:32:54.471088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:32:54.471088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:32:54.472250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:32:54.472250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:54.472250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:54.472250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:54.472250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:54.472250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:54.472250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:54.472250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:54.472250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:54.472250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:54.472250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:54.472250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:54.472250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:54.472250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:54.472250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:54.472250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:54.472250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:54.473288 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:32:54.473288 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:32:54.473288 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:32:54.473288 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:32:54.473288 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:32:54.473288 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:32:54.473288 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:32:54.473288 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:54.475509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 450/910] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:32:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:54.485338 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df479c8-54dd-4c01-b1db-cd6c7f9288ad None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:54.490371 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:54.656336 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df479c8-54dd-4c01-b1db-cd6c7f9288ad None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:54.711638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e438c-87e0-4ba7-a09f-229f799b3dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:54.712740 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e438c-87e0-4ba7-a09f-229f799b3dba None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:54.712740 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e438c-87e0-4ba7-a09f-229f799b3dba None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:54.712913 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e438c-87e0-4ba7-a09f-229f799b3dba None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:54.713544 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e438c-87e0-4ba7-a09f-229f799b3dba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:54.713846 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e438c-87e0-4ba7-a09f-229f799b3dba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:54.715319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e438c-87e0-4ba7-a09f-229f799b3dba None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:54.729335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 461/911] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:32:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:32:54.735722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df99317-9ea5-45b6-ae0d-88faff1fa468 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:54.871449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df99317-9ea5-45b6-ae0d-88faff1fa468 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:54.931861 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c8736-36ce-4d4e-a6dc-4dd16a30e227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:54.932413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c8736-36ce-4d4e-a6dc-4dd16a30e227 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:54.932750 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c8736-36ce-4d4e-a6dc-4dd16a30e227 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:54.933097 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c8736-36ce-4d4e-a6dc-4dd16a30e227 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:54.933886 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c8736-36ce-4d4e-a6dc-4dd16a30e227 None admin] RBAC: Adding query filter params (role.id=16940dcad5d247d0b43da26eb2b69cbc, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=9646662b0b3048f79ad5d2b9602f10a6) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:54.934194 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c8736-36ce-4d4e-a6dc-4dd16a30e227 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:54.936233 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c8736-36ce-4d4e-a6dc-4dd16a30e227 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:54.938686 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:54.944189 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 451/912] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:32:54 2022] GET /identity/v3/role_assignments?user.id=9646662b0b3048f79ad5d2b9602f10a6&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=16940dcad5d247d0b43da26eb2b69cbc => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:55.292056 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347e17a-6da3-4f07-bd31-4c2c9564c2f5 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:55.292759 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347e17a-6da3-4f07-bd31-4c2c9564c2f5 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:55.292759 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347e17a-6da3-4f07-bd31-4c2c9564c2f5 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:55.294123 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 462/913] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:55.303191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee074517-57cb-46f3-84bb-9f81f5700bfe None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:55.303903 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee074517-57cb-46f3-84bb-9f81f5700bfe None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:55.304420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee074517-57cb-46f3-84bb-9f81f5700bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:55.313295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3beb5-f69f-4c90-8fd0-57bce2a71238 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:55.313692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3beb5-f69f-4c90-8fd0-57bce2a71238 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:55.313846 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3beb5-f69f-4c90-8fd0-57bce2a71238 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:55.315525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 463/914] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:55.321091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546befd3-94ee-4b7a-b714-58ef69f76b0f None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:55.321311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546befd3-94ee-4b7a-b714-58ef69f76b0f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:55.321617 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546befd3-94ee-4b7a-b714-58ef69f76b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:55.339736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-ee074517-57cb-46f3-84bb-9f81f5700bfe None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:55.341433 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-546befd3-94ee-4b7a-b714-58ef69f76b0f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:55.341674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ee074517-57cb-46f3-84bb-9f81f5700bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:55.342894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-546befd3-94ee-4b7a-b714-58ef69f76b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:55.423404 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:55.430738 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 464/915] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:32:55.432039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 452/916] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:32:55.453014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546befd3-94ee-4b7a-b714-58ef69f76b0f None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:55.461885 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee074517-57cb-46f3-84bb-9f81f5700bfe None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:55.580505 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee074517-57cb-46f3-84bb-9f81f5700bfe None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:55.619235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:55.627873 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50fd8e-b653-4c14-831b-1986485f13b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:55.628531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50fd8e-b653-4c14-831b-1986485f13b7 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:55.634695 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50fd8e-b653-4c14-831b-1986485f13b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:55.634695 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50fd8e-b653-4c14-831b-1986485f13b7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:55.634695 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50fd8e-b653-4c14-831b-1986485f13b7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:55.634695 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50fd8e-b653-4c14-831b-1986485f13b7 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:55.637399 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546befd3-94ee-4b7a-b714-58ef69f76b0f None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:55.642313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-ac50fd8e-b653-4c14-831b-1986485f13b7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 16:32:55.642313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:55.642313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:32:55.642313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:32:55.642313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:32:55.642313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:32:55.642313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:32:55.642313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:55.642313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 16:32:55.642313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:55.642313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:32:55.642313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:55.642313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:55.642313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:55.642313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:55.642313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:55.642313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:55.643471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:55.643471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:55.643471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:55.643471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:55.643471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:55.643471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:55.643471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:32:55.643471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:32:55.643471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:32:55.643471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:32:55.643471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:55.643471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:55.643471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:55.643471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:55.643471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:55.643471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:55.644353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:55.644353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:55.644353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:55.644353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:55.644353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:55.644353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:55.644353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:55.644353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:55.644353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:55.644353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:55.644353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:32:55.644353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:55.644353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 16:32:55.644353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:55.649258 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 453/917] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Aug 8 16:32:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:55.651138 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50fd8e-b653-4c14-831b-1986485f13b7 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:55.684566 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcc98c-09a0-4acd-bc80-359149329595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:55.685569 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcc98c-09a0-4acd-bc80-359149329595 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:55.686048 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcc98c-09a0-4acd-bc80-359149329595 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:55.686504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcc98c-09a0-4acd-bc80-359149329595 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:55.687416 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fcc98c-09a0-4acd-bc80-359149329595 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:55.689080 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fcc98c-09a0-4acd-bc80-359149329595 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:55.703091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-22fcc98c-09a0-4acd-bc80-359149329595 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:32:55.703091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:55.703091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:32:55.703091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:32:55.703091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:32:55.703091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:32:55.703091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:32:55.703091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:55.703091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:32:55.703091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:55.703091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:32:55.703091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:55.703091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:55.703091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:55.703091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:55.703091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:55.703091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:55.704361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:55.704361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:55.704361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:55.704361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:55.704361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:55.704361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:55.704361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:32:55.704361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:32:55.704361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:32:55.704361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:32:55.704361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:55.704361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:55.704361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:55.704361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:55.704361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:55.704361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:55.705331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:55.705331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:55.705331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:55.705331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:55.705331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:55.705331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:55.705331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:55.705331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:55.705331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:55.705331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:55.705331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:32:55.705331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:55.705331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:32:55.705331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:55.708780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 465/918] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Aug 8 16:32:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:32:55.725811 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcc98c-09a0-4acd-bc80-359149329595 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:55.782870 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50fd8e-b653-4c14-831b-1986485f13b7 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:55.838525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:55.846471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbde716-87d8-4217-a9cf-9e8154130584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:55.847610 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbde716-87d8-4217-a9cf-9e8154130584 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:55.847996 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbde716-87d8-4217-a9cf-9e8154130584 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:55.848366 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbde716-87d8-4217-a9cf-9e8154130584 None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:55.849258 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbde716-87d8-4217-a9cf-9e8154130584 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:55.850056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbde716-87d8-4217-a9cf-9e8154130584 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:55.852926 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbde716-87d8-4217-a9cf-9e8154130584 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:55.866407 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 454/919] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Aug 8 16:32:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:55.868828 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbde716-87d8-4217-a9cf-9e8154130584 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:55.928042 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcc98c-09a0-4acd-bc80-359149329595 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:55.957792 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:55.975953 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62989ed-2ef1-483e-86c2-2b35fdb46d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:55.980129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62989ed-2ef1-483e-86c2-2b35fdb46d24 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:55.980129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62989ed-2ef1-483e-86c2-2b35fdb46d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:55.980129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62989ed-2ef1-483e-86c2-2b35fdb46d24 None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:55.980129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62989ed-2ef1-483e-86c2-2b35fdb46d24 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:55.980129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62989ed-2ef1-483e-86c2-2b35fdb46d24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:55.981003 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62989ed-2ef1-483e-86c2-2b35fdb46d24 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:55.990700 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 466/920] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Aug 8 16:32:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:55.993413 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62989ed-2ef1-483e-86c2-2b35fdb46d24 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:56.018188 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbde716-87d8-4217-a9cf-9e8154130584 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:56.088829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389917e-a7e7-4281-8fc6-ea050970fa35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:56.090804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389917e-a7e7-4281-8fc6-ea050970fa35 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:56.090804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389917e-a7e7-4281-8fc6-ea050970fa35 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:56.091026 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389917e-a7e7-4281-8fc6-ea050970fa35 None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:56.092419 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c389917e-a7e7-4281-8fc6-ea050970fa35 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:56.092961 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c389917e-a7e7-4281-8fc6-ea050970fa35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:56.095305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c389917e-a7e7-4281-8fc6-ea050970fa35 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:56.110787 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 455/921] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Aug 8 16:32:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:32:56.121596 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389917e-a7e7-4281-8fc6-ea050970fa35 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:56.132545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62989ed-2ef1-483e-86c2-2b35fdb46d24 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:56.160279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:56.179116 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10abab1-405d-4a08-98cd-063eb1a5f2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:56.181323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10abab1-405d-4a08-98cd-063eb1a5f2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:56.182656 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10abab1-405d-4a08-98cd-063eb1a5f2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:56.183443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10abab1-405d-4a08-98cd-063eb1a5f2b0 None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:56.184476 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10abab1-405d-4a08-98cd-063eb1a5f2b0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:56.185092 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10abab1-405d-4a08-98cd-063eb1a5f2b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:56.187659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10abab1-405d-4a08-98cd-063eb1a5f2b0 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:56.197459 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 467/922] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Aug 8 16:32:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:32:56.208007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10abab1-405d-4a08-98cd-063eb1a5f2b0 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:56.309597 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389917e-a7e7-4281-8fc6-ea050970fa35 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:56.333267 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:56.361065 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae232dae-6ebc-45f3-8512-59d9b715cfd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:56.362116 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae232dae-6ebc-45f3-8512-59d9b715cfd9 None admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:56.362699 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae232dae-6ebc-45f3-8512-59d9b715cfd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:56.363142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae232dae-6ebc-45f3-8512-59d9b715cfd9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:56.364365 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae232dae-6ebc-45f3-8512-59d9b715cfd9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:56.367609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae232dae-6ebc-45f3-8512-59d9b715cfd9 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:56.374144 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10abab1-405d-4a08-98cd-063eb1a5f2b0 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:56.439645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c43e5e-2d24-492a-8408-4df5bc7e2434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:56.440449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c43e5e-2d24-492a-8408-4df5bc7e2434 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:56.440843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c43e5e-2d24-492a-8408-4df5bc7e2434 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:56.441239 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c43e5e-2d24-492a-8408-4df5bc7e2434 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:56.442177 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c43e5e-2d24-492a-8408-4df5bc7e2434 None admin] RBAC: Adding query filter params (interface=public, service_id=52005eaa9dce406bafbba660d82e59f0, region_id=RegionOne) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:56.442654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c43e5e-2d24-492a-8408-4df5bc7e2434 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:56.444314 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c43e5e-2d24-492a-8408-4df5bc7e2434 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:56.451704 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 468/923] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Aug 8 16:32:56 2022] GET /identity/v3/endpoints?service_id=52005eaa9dce406bafbba660d82e59f0&interface=public®ion_id=RegionOne => generated 194 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:32:56.453962 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf35932-b8cd-4cc6-bba3-29913e8253a3 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:56.454442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf35932-b8cd-4cc6-bba3-29913e8253a3 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:56.454939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf35932-b8cd-4cc6-bba3-29913e8253a3 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:56.457784 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 469/924] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:56.461691 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf35932-b8cd-4cc6-bba3-29913e8253a3 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:56.525923 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 456/925] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:32:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:32:56.528552 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877eb7e3-200e-474b-b091-f1c53a35bdb6 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:56.529128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877eb7e3-200e-474b-b091-f1c53a35bdb6 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:56.529637 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877eb7e3-200e-474b-b091-f1c53a35bdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:56.551862 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-877eb7e3-200e-474b-b091-f1c53a35bdb6 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:56.553320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:56.554444 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-877eb7e3-200e-474b-b091-f1c53a35bdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:56.700797 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf35932-b8cd-4cc6-bba3-29913e8253a3 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:56.711244 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 457/926] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:32:56.749190 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877eb7e3-200e-474b-b091-f1c53a35bdb6 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:56.771050 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d38b17-dd37-4b6b-b9b6-65e203d7207b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:56.772100 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d38b17-dd37-4b6b-b9b6-65e203d7207b None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:56.772594 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d38b17-dd37-4b6b-b9b6-65e203d7207b None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:56.773050 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d38b17-dd37-4b6b-b9b6-65e203d7207b None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:56.774245 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d38b17-dd37-4b6b-b9b6-65e203d7207b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:56.776895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d38b17-dd37-4b6b-b9b6-65e203d7207b None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:56.788505 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 470/927] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Aug 8 16:32:56 2022] GET /identity/v3/services => generated 1497 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:32:56.971101 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877eb7e3-200e-474b-b091-f1c53a35bdb6 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:56.974084 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:57.038036 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b481c-ac05-4805-a9d3-11681de70933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:57.039463 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b481c-ac05-4805-a9d3-11681de70933 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:57.039935 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b481c-ac05-4805-a9d3-11681de70933 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:57.040397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b481c-ac05-4805-a9d3-11681de70933 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:57.041446 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b481c-ac05-4805-a9d3-11681de70933 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:57.044030 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b481c-ac05-4805-a9d3-11681de70933 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:57.062178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-a40b481c-ac05-4805-a9d3-11681de70933 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 16:32:57.062178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:57.062178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:32:57.062178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:32:57.062178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:32:57.062178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:32:57.062178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:32:57.062178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:57.062178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 16:32:57.062178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:57.062178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:32:57.062178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:57.062178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:57.062178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:57.062178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:57.062178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:57.063317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:57.063317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:57.063317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:57.063317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:57.063317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:57.063317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:57.063317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:57.063317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:32:57.063317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:32:57.063317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:32:57.063317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:32:57.063317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:57.063317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:57.063317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:57.063317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:57.063317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:57.063317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:57.064264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:57.064264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:57.064264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:57.064264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:57.064264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:57.064264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:57.064264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:57.064264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:57.064264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:57.064264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:57.064264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:32:57.064264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:57.064264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 16:32:57.064264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:57.068307 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 458/928] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Aug 8 16:32:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:57.074396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d38b17-dd37-4b6b-b9b6-65e203d7207b None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:57.216054 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d38b17-dd37-4b6b-b9b6-65e203d7207b None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:57.228361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:57.256365 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e89b8-9f44-416f-85fe-a01028fdc94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:57.257385 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e89b8-9f44-416f-85fe-a01028fdc94d None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:57.257929 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e89b8-9f44-416f-85fe-a01028fdc94d None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:57.259119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e89b8-9f44-416f-85fe-a01028fdc94d None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:57.259993 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e89b8-9f44-416f-85fe-a01028fdc94d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:57.260506 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e89b8-9f44-416f-85fe-a01028fdc94d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:57.262145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e89b8-9f44-416f-85fe-a01028fdc94d None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:57.268287 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 471/929] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Aug 8 16:32:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:57.279117 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b481c-ac05-4805-a9d3-11681de70933 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:57.400047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b481c-ac05-4805-a9d3-11681de70933 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:57.440421 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b1c02-8991-4ae9-a261-a2de9ea76ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:57.441369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b1c02-8991-4ae9-a261-a2de9ea76ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:57.442738 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b1c02-8991-4ae9-a261-a2de9ea76ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:57.442738 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b1c02-8991-4ae9-a261-a2de9ea76ac7 None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:57.442738 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b1c02-8991-4ae9-a261-a2de9ea76ac7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:57.443014 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b1c02-8991-4ae9-a261-a2de9ea76ac7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:57.444435 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b1c02-8991-4ae9-a261-a2de9ea76ac7 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:57.450739 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 459/930] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Aug 8 16:32:57 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:57.459547 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e89b8-9f44-416f-85fe-a01028fdc94d None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:57.515174 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e47416-31f6-4214-a42f-5624db623465 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:57.515666 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e47416-31f6-4214-a42f-5624db623465 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:57.515954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e47416-31f6-4214-a42f-5624db623465 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:57.517897 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 460/931] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:32:57.523296 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593eaa1-7b8a-4532-8489-d52785ca0c35 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:57.524245 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593eaa1-7b8a-4532-8489-d52785ca0c35 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:57.524515 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593eaa1-7b8a-4532-8489-d52785ca0c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:57.544879 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-b593eaa1-7b8a-4532-8489-d52785ca0c35 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:57.546122 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-b593eaa1-7b8a-4532-8489-d52785ca0c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:57.583234 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:57.613886 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e89b8-9f44-416f-85fe-a01028fdc94d None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:57.662239 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a153f-ee7b-49f2-8c28-6e41d545f655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:57.662907 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a153f-ee7b-49f2-8c28-6e41d545f655 None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:57.663185 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a153f-ee7b-49f2-8c28-6e41d545f655 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:57.663460 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a153f-ee7b-49f2-8c28-6e41d545f655 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:57.664235 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a153f-ee7b-49f2-8c28-6e41d545f655 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:57.665676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a153f-ee7b-49f2-8c28-6e41d545f655 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:57.682743 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 461/932] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:32:57.717691 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593eaa1-7b8a-4532-8489-d52785ca0c35 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:57.802871 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 472/933] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:32:57 2022] POST /identity/v3/endpoints => generated 347 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:32:57.883529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593eaa1-7b8a-4532-8489-d52785ca0c35 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:57.961978 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7be06-6f73-4d8a-a85d-b3ec895042d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:57.963063 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7be06-6f73-4d8a-a85d-b3ec895042d3 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:57.963531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7be06-6f73-4d8a-a85d-b3ec895042d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:57.963995 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7be06-6f73-4d8a-a85d-b3ec895042d3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:57.985370 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e7be06-6f73-4d8a-a85d-b3ec895042d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:32:57.995151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-18e7be06-6f73-4d8a-a85d-b3ec895042d3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:32:57.995151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:57.995151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:57.995151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:57.995151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:57.995151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:57.995151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:57.995151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:57.995151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:57.995151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:57.995151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:57.995151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:57.995151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:32:57.995151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:32:57.995151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:32:57.995151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:32:57.997051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:32:57.997051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:32:57.997051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:32:57.997051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:32:57.997051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:57.997051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:57.997051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:57.997051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:57.997051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:57.997051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:57.997051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:57.997051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:57.997051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:57.997051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:57.997051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:57.997051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:58.000491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:58.000491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:58.000491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:58.000491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:58.000491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:32:58.000491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:32:58.000491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:32:58.000491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:32:58.000491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:32:58.000491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:32:58.000491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:32:58.000491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:58.000491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 462/934] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Aug 8 16:32:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:32:58.007223 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a153f-ee7b-49f2-8c28-6e41d545f655 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:58.068379 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:58.196712 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a153f-ee7b-49f2-8c28-6e41d545f655 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:58.273089 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e835b0-6206-4cee-a992-eafcf6192e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:58.274200 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e835b0-6206-4cee-a992-eafcf6192e57 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:58.274746 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e835b0-6206-4cee-a992-eafcf6192e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:58.275224 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e835b0-6206-4cee-a992-eafcf6192e57 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:58.276452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e835b0-6206-4cee-a992-eafcf6192e57 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:58.276987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e835b0-6206-4cee-a992-eafcf6192e57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:58.279710 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e835b0-6206-4cee-a992-eafcf6192e57 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:58.296592 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 473/935] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Aug 8 16:32:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:58.306591 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7be06-6f73-4d8a-a85d-b3ec895042d3 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:58.346392 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:58.518781 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7be06-6f73-4d8a-a85d-b3ec895042d3 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:58.586642 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79c135-3425-44c5-8f47-a0a7e0a2db04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:58.587713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79c135-3425-44c5-8f47-a0a7e0a2db04 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:58.588186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79c135-3425-44c5-8f47-a0a7e0a2db04 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:58.588653 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79c135-3425-44c5-8f47-a0a7e0a2db04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:58.609967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79c135-3425-44c5-8f47-a0a7e0a2db04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:58.612560 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79c135-3425-44c5-8f47-a0a7e0a2db04 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:58.642150 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-4d79c135-3425-44c5-8f47-a0a7e0a2db04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:58.642150 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:58.642150 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:58.642150 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:58.642150 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:58.642150 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:58.642150 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:58.642150 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:58.642150 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:58.642150 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:58.642150 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:58.642150 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:58.642150 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:58.642150 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:58.642150 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:58.642150 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:58.643319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:58.643319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:58.643319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:58.643319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:58.643319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:58.643319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:58.643319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:58.643319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:58.643319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:58.643319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:58.643319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:58.643319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:58.643319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:58.643319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:58.643319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:58.643319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:58.643319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:58.644244 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:58.644244 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:58.644244 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:58.644950 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 463/936] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:58.653979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e835b0-6206-4cee-a992-eafcf6192e57 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:58.829396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:58.850883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e835b0-6206-4cee-a992-eafcf6192e57 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:58.903831 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17cd8c-c0ed-4ea9-957a-b5f8ba607636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:58.904527 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17cd8c-c0ed-4ea9-957a-b5f8ba607636 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:58.904848 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17cd8c-c0ed-4ea9-957a-b5f8ba607636 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:58.905373 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17cd8c-c0ed-4ea9-957a-b5f8ba607636 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:58.906596 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a17cd8c-c0ed-4ea9-957a-b5f8ba607636 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:58.907109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a17cd8c-c0ed-4ea9-957a-b5f8ba607636 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:58.908762 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a17cd8c-c0ed-4ea9-957a-b5f8ba607636 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:58.929026 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 474/937] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:58.936343 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79c135-3425-44c5-8f47-a0a7e0a2db04 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:59.047943 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:59.112631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79c135-3425-44c5-8f47-a0a7e0a2db04 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:59.179375 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60060140-bca8-4ed0-8263-ae3b62f38df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:59.180392 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60060140-bca8-4ed0-8263-ae3b62f38df2 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:59.181338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60060140-bca8-4ed0-8263-ae3b62f38df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:59.181808 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60060140-bca8-4ed0-8263-ae3b62f38df2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:59.197175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60060140-bca8-4ed0-8263-ae3b62f38df2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:59.199943 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60060140-bca8-4ed0-8263-ae3b62f38df2 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:59.223178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-60060140-bca8-4ed0-8263-ae3b62f38df2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:32:59.223178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:59.223178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:59.223178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:59.223178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:59.223178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:59.223178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:59.223178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:59.223178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:59.223178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:59.223178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:59.223178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:59.223178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:32:59.223178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:32:59.223178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:32:59.223178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:32:59.225034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:59.225034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:59.225034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:32:59.225034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:59.225034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:32:59.225034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:32:59.225034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:59.225034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:59.225034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:59.225034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:59.225034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:59.225034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:59.225034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:59.225034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:59.225034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:59.225034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:59.225034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:59.225034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:59.228208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:59.228208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:59.228208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:32:59.228208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:32:59.228208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:32:59.228208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:32:59.228208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:32:59.228208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:32:59.228208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:32:59.228208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:59.228208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 464/938] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Aug 8 16:32:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:32:59.236501 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17cd8c-c0ed-4ea9-957a-b5f8ba607636 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:59.399563 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c15b2-83cd-4841-a1cb-2666d9f768bd None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:59.400874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c15b2-83cd-4841-a1cb-2666d9f768bd None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:59.401706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c15b2-83cd-4841-a1cb-2666d9f768bd None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:59.406061 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 465/939] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:59.416223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e164ae9-dbd5-4d1f-b097-b435906d27ff None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:59.417106 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e164ae9-dbd5-4d1f-b097-b435906d27ff None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:59.417885 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e164ae9-dbd5-4d1f-b097-b435906d27ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:59.446380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17cd8c-c0ed-4ea9-957a-b5f8ba607636 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:59.450269 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-8e164ae9-dbd5-4d1f-b097-b435906d27ff None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:59.452776 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-8e164ae9-dbd5-4d1f-b097-b435906d27ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:32:59.517069 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c0774-2784-4e4e-9824-f8635d4ae3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:59.518458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c0774-2784-4e4e-9824-f8635d4ae3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:59.519172 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c0774-2784-4e4e-9824-f8635d4ae3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:59.519836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c0774-2784-4e4e-9824-f8635d4ae3d9 None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:59.521703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c0774-2784-4e4e-9824-f8635d4ae3d9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:32:59.522476 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c0774-2784-4e4e-9824-f8635d4ae3d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:59.525616 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c0774-2784-4e4e-9824-f8635d4ae3d9 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:59.563110 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 475/940] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Aug 8 16:32:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:32:59.574680 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c0774-2784-4e4e-9824-f8635d4ae3d9 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:59.596086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:59.599069 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 466/941] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:32:59.634016 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e164ae9-dbd5-4d1f-b097-b435906d27ff None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:59.735344 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:32:59.766036 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c0774-2784-4e4e-9824-f8635d4ae3d9 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:59.795333 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e164ae9-dbd5-4d1f-b097-b435906d27ff None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:32:59.807563 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e7979-8ca8-410f-a4c4-1af1a642b592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:59.808230 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e7979-8ca8-410f-a4c4-1af1a642b592 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:59.808505 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e7979-8ca8-410f-a4c4-1af1a642b592 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:59.808772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e7979-8ca8-410f-a4c4-1af1a642b592 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:59.821519 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690e7979-8ca8-410f-a4c4-1af1a642b592 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:59.823079 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690e7979-8ca8-410f-a4c4-1af1a642b592 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:59.850686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-690e7979-8ca8-410f-a4c4-1af1a642b592 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:59.850686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:59.850686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:59.850686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:59.850686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:59.850686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:59.850686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:59.850686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:59.850686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:59.850686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:59.850686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:59.850686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:59.850686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:32:59.850686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:32:59.850686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:32:59.850686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:32:59.854981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:59.854981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:59.854981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:59.854981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:59.854981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:59.854981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:59.854981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:59.854981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:59.854981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:59.854981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:59.854981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:59.854981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:59.854981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:59.854981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:59.854981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:59.854981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:59.854981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:32:59.856736 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:32:59.856736 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:32:59.856736 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:32:59.856736 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 476/942] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:32:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:59.856736 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3f272-3941-446c-a3c2-93c0190f85d2 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:59.856736 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3f272-3941-446c-a3c2-93c0190f85d2 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:59.857408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3f272-3941-446c-a3c2-93c0190f85d2 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:59.859643 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 477/943] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:32:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:32:59.867158 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3f272-3941-446c-a3c2-93c0190f85d2 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:32:59.879043 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cde595-c29b-481b-af8b-0406165480b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:32:59.880062 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cde595-c29b-481b-af8b-0406165480b5 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:59.880524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cde595-c29b-481b-af8b-0406165480b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:59.880959 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cde595-c29b-481b-af8b-0406165480b5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:59.881952 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cde595-c29b-481b-af8b-0406165480b5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:32:59.885547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cde595-c29b-481b-af8b-0406165480b5 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:32:59.906714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-77cde595-c29b-481b-af8b-0406165480b5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:32:59.906714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:59.906714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:32:59.906714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:32:59.906714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:32:59.906714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:32:59.906714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:32:59.906714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:59.906714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:32:59.906714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:59.906714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:32:59.906714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:59.906714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:32:59.906714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:32:59.906714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:32:59.906714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:32:59.906714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:32:59.908815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:32:59.908815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:32:59.908815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:32:59.908815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:32:59.908815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:32:59.908815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:32:59.908815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:32:59.908815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:32:59.908815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:32:59.908815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:32:59.908815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:32:59.908815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:32:59.908815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:32:59.908815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:32:59.908815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:32:59.908815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:32:59.910559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:32:59.910559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:32:59.910559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:32:59.910559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:32:59.910559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:32:59.910559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:32:59.910559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:32:59.910559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:32:59.910559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:32:59.910559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:32:59.910559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:32:59.910559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:32:59.910559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:32:59.910559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:32:59.912183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 467/944] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Aug 8 16:32:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:32:59.913261 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad7ec3-f564-4790-b414-6a4f1a264d74 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:32:59.913516 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad7ec3-f564-4790-b414-6a4f1a264d74 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:32:59.914103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad7ec3-f564-4790-b414-6a4f1a264d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:32:59.945714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-b8ad7ec3-f564-4790-b414-6a4f1a264d74 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:32:59.947358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-b8ad7ec3-f564-4790-b414-6a4f1a264d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:00.007306 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:00.052821 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 468/945] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:32:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:33:00.054203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad7ec3-f564-4790-b414-6a4f1a264d74 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:00.096544 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3f272-3941-446c-a3c2-93c0190f85d2 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:00.156109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba018a9-dae3-440c-ac18-341ca60984f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:00.156937 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:00.157757 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba018a9-dae3-440c-ac18-341ca60984f6 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:00.158226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba018a9-dae3-440c-ac18-341ca60984f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:00.158594 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba018a9-dae3-440c-ac18-341ca60984f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:00.159626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba018a9-dae3-440c-ac18-341ca60984f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:00.160130 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba018a9-dae3-440c-ac18-341ca60984f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:00.162282 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba018a9-dae3-440c-ac18-341ca60984f6 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:00.178298 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 478/946] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:00.179187 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba018a9-dae3-440c-ac18-341ca60984f6 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:00.243575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad7ec3-f564-4790-b414-6a4f1a264d74 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:00.282338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d03ed2-f30c-4add-a613-42e33b6aa11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:00.282909 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d03ed2-f30c-4add-a613-42e33b6aa11e None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:00.283186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d03ed2-f30c-4add-a613-42e33b6aa11e None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:00.283456 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d03ed2-f30c-4add-a613-42e33b6aa11e None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:00.284098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d03ed2-f30c-4add-a613-42e33b6aa11e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:00.284391 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d03ed2-f30c-4add-a613-42e33b6aa11e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:00.285780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d03ed2-f30c-4add-a613-42e33b6aa11e None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:00.293120 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 469/947] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Aug 8 16:33:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:00.294352 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d03ed2-f30c-4add-a613-42e33b6aa11e None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:00.314474 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba018a9-dae3-440c-ac18-341ca60984f6 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:00.340337 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:00.359024 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c21d4-b7e2-4278-bd33-f8a4c8e08f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:00.359865 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c21d4-b7e2-4278-bd33-f8a4c8e08f95 None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:00.360238 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c21d4-b7e2-4278-bd33-f8a4c8e08f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:00.360569 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c21d4-b7e2-4278-bd33-f8a4c8e08f95 None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:00.361427 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c21d4-b7e2-4278-bd33-f8a4c8e08f95 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:00.363346 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c21d4-b7e2-4278-bd33-f8a4c8e08f95 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:00.368527 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: WARNING keystone.server.flask.common [None req-763c21d4-b7e2-4278-bd33-f8a4c8e08f95 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 08 16:33:00.410364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d03ed2-f30c-4add-a613-42e33b6aa11e None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:00.430199 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:00.449883 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6cc4da-931a-464c-94cf-8fabcc8b4891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:00.450525 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cc4da-931a-464c-94cf-8fabcc8b4891 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:00.450833 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cc4da-931a-464c-94cf-8fabcc8b4891 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:00.451095 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cc4da-931a-464c-94cf-8fabcc8b4891 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:00.456656 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.sql.core [None req-763c21d4-b7e2-4278-bd33-f8a4c8e08f95 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 16:33:00.456656 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 08 16:33:00.456656 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [parameters: {'id': '8d0e68a28c5d481f868da6c36551c962', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 08 16:33:00.456656 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81999) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 16:33:00.459727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6cc4da-931a-464c-94cf-8fabcc8b4891 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:00.462655 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6cc4da-931a-464c-94cf-8fabcc8b4891 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:00.467683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-763c21d4-b7e2-4278-bd33-f8a4c8e08f95 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8d0e68a28c5d481f868da6c36551c962.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8d0e68a28c5d481f868da6c36551c962. Aug 08 16:33:00.467683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:00.467683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 16:33:00.467683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 16:33:00.467683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 16:33:00.467683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 16:33:00.467683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 16:33:00.467683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 16:33:00.467683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 16:33:00.467683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application conn.query(q) Aug 08 16:33:00.467683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 16:33:00.467683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 16:33:00.467683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 16:33:00.469319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application result.read() Aug 08 16:33:00.469319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 16:33:00.469319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 16:33:00.469319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 16:33:00.469319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 16:33:00.469319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 16:33:00.469319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 16:33:00.469319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 16:33:00.469319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 16:33:00.469319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 16:33:00.469319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:00.469319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 16:33:00.469319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:00.469319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:00.469319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 16:33:00.469319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 16:33:00.469319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 08 16:33:00.469319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 08 16:33:00.471260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 16:33:00.471260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application next(self.gen) Aug 08 16:33:00.471260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 16:33:00.471260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application yield resource Aug 08 16:33:00.471260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 16:33:00.471260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application next(self.gen) Aug 08 16:33:00.471260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 16:33:00.471260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 16:33:00.471260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 16:33:00.471260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 16:33:00.471260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 16:33:00.471260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise self.value Aug 08 16:33:00.471260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 16:33:00.471260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 16:33:00.471260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 16:33:00.471260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application session.commit() Aug 08 16:33:00.471260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 16:33:00.472994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 16:33:00.472994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 16:33:00.472994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 16:33:00.472994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 16:33:00.472994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self.session.flush() Aug 08 16:33:00.472994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 16:33:00.472994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 16:33:00.472994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 16:33:00.472994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 16:33:00.472994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 16:33:00.472994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application compat.raise_( Aug 08 16:33:00.472994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 16:33:00.472994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception Aug 08 16:33:00.472994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 16:33:00.472994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 16:33:00.472994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 16:33:00.472994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 16:33:00.475713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 16:33:00.475713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 16:33:00.475713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 16:33:00.475713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 16:33:00.475713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 08 16:33:00.475713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 08 16:33:00.475713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 16:33:00.475713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 16:33:00.475713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 16:33:00.475713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 16:33:00.475713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 16:33:00.475713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 16:33:00.475713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 16:33:00.475713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 16:33:00.475713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 16:33:00.478199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 16:33:00.478199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 16:33:00.478199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception Aug 08 16:33:00.478199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 16:33:00.478199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 16:33:00.478199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 16:33:00.478199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 16:33:00.478199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 16:33:00.478199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 16:33:00.478199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 16:33:00.478199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application conn.query(q) Aug 08 16:33:00.478199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 16:33:00.478199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 16:33:00.478199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 16:33:00.478199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application result.read() Aug 08 16:33:00.478199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 16:33:00.478199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 16:33:00.480445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 16:33:00.480445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 16:33:00.480445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 16:33:00.480445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 16:33:00.480445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 16:33:00.480445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 16:33:00.480445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 16:33:00.480445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 08 16:33:00.480445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [parameters: {'id': '8d0e68a28c5d481f868da6c36551c962', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 08 16:33:00.480445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 16:33:00.480445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:00.480445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:33:00.480445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:00.480445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:00.480445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 08 16:33:00.482049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 08 16:33:00.482049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 16:33:00.482049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 16:33:00.482049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 08 16:33:00.482049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:00.482049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:33:00.482049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:00.482049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:00.482049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:00.482049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:00.482049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:00.482049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:00.482049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:00.482049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:00.482049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:00.482049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:00.482049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:00.484561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:00.484561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 08 16:33:00.484561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 08 16:33:00.484561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:00.484561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:00.484561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 08 16:33:00.484561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 08 16:33:00.484561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8d0e68a28c5d481f868da6c36551c962. Aug 08 16:33:00.484561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:00.484561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 479/948] 10.222.0.13 () {60 vars in 1140 bytes} [Mon Aug 8 16:33:00 2022] POST /identity/v3/projects => generated 262 bytes in 290 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:00.484561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ffad0-ce9d-4b5c-9481-f32a1d138eeb None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:00.484561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ffad0-ce9d-4b5c-9481-f32a1d138eeb None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:00.486871 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ffad0-ce9d-4b5c-9481-f32a1d138eeb None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:00.487013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-bd6cc4da-931a-464c-94cf-8fabcc8b4891 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:33:00.487013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:00.487013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:00.487013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:00.487013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:00.487013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:00.487013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:00.487013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:00.487013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:00.487013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:00.487013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:00.487013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:00.487013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:33:00.487013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:33:00.487013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:33:00.488919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:33:00.488919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:00.488919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:00.488919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:00.488919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:00.488919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:00.488919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:00.488919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:00.488919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:00.488919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:00.488919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:00.488919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:00.488919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:00.488919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:00.488919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:00.488919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:00.488919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:00.491850 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:33:00.491850 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:33:00.491850 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:33:00.491850 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:33:00.491850 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:33:00.491850 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:33:00.491850 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:33:00.491850 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:00.493416 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 480/949] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:00 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:33:00.493583 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 470/950] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:33:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:00.493583 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6cc4da-931a-464c-94cf-8fabcc8b4891 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:00.493845 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ffad0-ce9d-4b5c-9481-f32a1d138eeb None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:00.619206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ffad0-ce9d-4b5c-9481-f32a1d138eeb None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:00.628813 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6cc4da-931a-464c-94cf-8fabcc8b4891 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:00.671156 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e4eaf3-777a-46b3-99cd-8ae1559a4a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:00.671827 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4eaf3-777a-46b3-99cd-8ae1559a4a56 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:00.672106 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4eaf3-777a-46b3-99cd-8ae1559a4a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:00.672376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4eaf3-777a-46b3-99cd-8ae1559a4a56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:00.675265 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae2459-f50e-479e-99ff-ca51008dc708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:00.676298 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae2459-f50e-479e-99ff-ca51008dc708 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:00.676793 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae2459-f50e-479e-99ff-ca51008dc708 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:00.677291 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae2459-f50e-479e-99ff-ca51008dc708 None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:00.678486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae2459-f50e-479e-99ff-ca51008dc708 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:00.679064 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae2459-f50e-479e-99ff-ca51008dc708 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:00.682434 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae2459-f50e-479e-99ff-ca51008dc708 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:00.684228 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e4eaf3-777a-46b3-99cd-8ae1559a4a56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:00.687111 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e4eaf3-777a-46b3-99cd-8ae1559a4a56 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:00.691991 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 481/951] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Aug 8 16:33:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:00.694256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bbd9f-70b1-4595-8917-c850ac5ea8f4 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:00.694834 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bbd9f-70b1-4595-8917-c850ac5ea8f4 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:00.695428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bbd9f-70b1-4595-8917-c850ac5ea8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:00.709878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-24e4eaf3-777a-46b3-99cd-8ae1559a4a56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:33:00.709878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:00.709878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:00.709878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:00.709878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:00.709878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:00.709878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:00.709878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:00.709878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:00.709878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:00.709878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:00.709878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:00.709878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:33:00.709878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:33:00.709878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:33:00.711541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:33:00.711541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:00.711541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:00.711541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:00.711541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:00.711541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:00.711541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:00.711541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:00.711541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:00.711541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:00.711541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:00.711541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:00.711541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:00.711541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:00.711541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:00.711541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:00.711541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:00.713296 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:33:00.713296 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:33:00.713296 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:33:00.713296 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:33:00.713296 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:33:00.713296 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:33:00.713296 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:33:00.713296 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:00.713296 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 471/952] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:33:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:00.714630 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e4eaf3-777a-46b3-99cd-8ae1559a4a56 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:00.727934 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-fc8bbd9f-70b1-4595-8917-c850ac5ea8f4 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:00.730781 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-fc8bbd9f-70b1-4595-8917-c850ac5ea8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:00.777734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:00.793676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:00.831243 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 482/953] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:33:00.832288 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8bbd9f-70b1-4595-8917-c850ac5ea8f4 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:00.892028 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e4eaf3-777a-46b3-99cd-8ae1559a4a56 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:00.943523 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e5967-9b56-4489-a489-3b0344304bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:00.944276 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e5967-9b56-4489-a489-3b0344304bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:00.944626 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e5967-9b56-4489-a489-3b0344304bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:00.944965 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e5967-9b56-4489-a489-3b0344304bb6 None admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:00.945770 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e5967-9b56-4489-a489-3b0344304bb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:00.946139 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e5967-9b56-4489-a489-3b0344304bb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:00.947801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e5967-9b56-4489-a489-3b0344304bb6 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:00.972041 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 472/954] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:33:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:00.972041 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e5967-9b56-4489-a489-3b0344304bb6 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:00.985911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8bbd9f-70b1-4595-8917-c850ac5ea8f4 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:01.041921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c811afc-383d-45da-bdea-0f1112f8330b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:01.042658 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c811afc-383d-45da-bdea-0f1112f8330b None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:01.043004 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c811afc-383d-45da-bdea-0f1112f8330b None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:01.043337 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c811afc-383d-45da-bdea-0f1112f8330b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:01.044132 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c811afc-383d-45da-bdea-0f1112f8330b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:01.045770 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c811afc-383d-45da-bdea-0f1112f8330b None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:01.102640 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:01.108850 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e5967-9b56-4489-a489-3b0344304bb6 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:01.127460 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 483/955] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:33:00 2022] POST /identity/v3/endpoints => generated 349 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:33:01.128321 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c811afc-383d-45da-bdea-0f1112f8330b None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:01.169891 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73355b58-32d1-486b-80bc-a21f6e14af1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:01.170855 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73355b58-32d1-486b-80bc-a21f6e14af1f None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:01.171305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73355b58-32d1-486b-80bc-a21f6e14af1f None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:01.171802 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73355b58-32d1-486b-80bc-a21f6e14af1f None admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:01.172913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73355b58-32d1-486b-80bc-a21f6e14af1f None admin] RBAC: Adding query filter params (name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:01.173275 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73355b58-32d1-486b-80bc-a21f6e14af1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:01.175874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73355b58-32d1-486b-80bc-a21f6e14af1f None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:01.183787 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:01.217846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 473/956] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Aug 8 16:33:00 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:01.217846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73355b58-32d1-486b-80bc-a21f6e14af1f None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:01.309913 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c811afc-383d-45da-bdea-0f1112f8330b None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:01.356608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46226ca6-9937-45ad-95d1-b69c8ce2b520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:01.357275 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46226ca6-9937-45ad-95d1-b69c8ce2b520 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:01.357554 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46226ca6-9937-45ad-95d1-b69c8ce2b520 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:01.357836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46226ca6-9937-45ad-95d1-b69c8ce2b520 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:01.358921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46226ca6-9937-45ad-95d1-b69c8ce2b520 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:01.360781 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46226ca6-9937-45ad-95d1-b69c8ce2b520 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:01.366866 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73355b58-32d1-486b-80bc-a21f6e14af1f None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:01.371638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-46226ca6-9937-45ad-95d1-b69c8ce2b520 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 16:33:01.371638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:01.371638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:33:01.371638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:33:01.371638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:33:01.371638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:33:01.371638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:33:01.371638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:33:01.371638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 16:33:01.371638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:01.371638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:33:01.371638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:01.371638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:01.371638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:01.371638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:01.371638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:01.371638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:01.373833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:01.373833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:01.373833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:01.373833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:01.373833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:01.373833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:01.373833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:33:01.373833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:33:01.373833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:33:01.373833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:33:01.373833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:01.373833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:01.373833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:01.373833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:01.373833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:01.373833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:01.375568 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:01.375568 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:01.375568 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:01.375568 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:01.375568 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:01.375568 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:01.375568 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:01.375568 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:01.375568 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:01.375568 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:01.375568 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:33:01.375568 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:33:01.375568 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 16:33:01.375568 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:01.375568 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 484/957] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:33:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:01.384969 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46226ca6-9937-45ad-95d1-b69c8ce2b520 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:01.416423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d7291-fdfe-4945-9208-be9188a1c6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:01.417996 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d7291-fdfe-4945-9208-be9188a1c6f7 None admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:01.419281 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d7291-fdfe-4945-9208-be9188a1c6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:01.419780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d7291-fdfe-4945-9208-be9188a1c6f7 None admin] PATH_INFO: `/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/9646662b0b3048f79ad5d2b9602f10a6/roles/16940dcad5d247d0b43da26eb2b69cbc` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:01.424033 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d7291-fdfe-4945-9208-be9188a1c6f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=56ab2fc79d964f4d86edefb4d1b2c993, user_id=9646662b0b3048f79ad5d2b9602f10a6, role_id=16940dcad5d247d0b43da26eb2b69cbc)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:01.428602 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d7291-fdfe-4945-9208-be9188a1c6f7 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:01.515971 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:01.525984 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 474/958] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:33:01 2022] PUT /identity/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/9646662b0b3048f79ad5d2b9602f10a6/roles/16940dcad5d247d0b43da26eb2b69cbc => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:33:01.535616 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46226ca6-9937-45ad-95d1-b69c8ce2b520 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:01.584720 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24227b6b-2bde-4ade-ac88-c139b5bdbda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:01.585374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24227b6b-2bde-4ade-ac88-c139b5bdbda9 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:01.585669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24227b6b-2bde-4ade-ac88-c139b5bdbda9 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:01.585972 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24227b6b-2bde-4ade-ac88-c139b5bdbda9 None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:01.587154 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24227b6b-2bde-4ade-ac88-c139b5bdbda9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:01.587311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24227b6b-2bde-4ade-ac88-c139b5bdbda9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:01.588769 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24227b6b-2bde-4ade-ac88-c139b5bdbda9 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:01.598784 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 485/959] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Aug 8 16:33:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:01.603606 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d7291-fdfe-4945-9208-be9188a1c6f7 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:01.636499 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:01.805857 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d7291-fdfe-4945-9208-be9188a1c6f7 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:01.870689 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e822a8-0bed-44e3-a88e-3b049506295d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:01.871313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e822a8-0bed-44e3-a88e-3b049506295d None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:01.871585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e822a8-0bed-44e3-a88e-3b049506295d None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:01.871850 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e822a8-0bed-44e3-a88e-3b049506295d None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:01.872484 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e822a8-0bed-44e3-a88e-3b049506295d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:01.872781 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e822a8-0bed-44e3-a88e-3b049506295d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:01.874242 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e822a8-0bed-44e3-a88e-3b049506295d None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:01.880213 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 475/960] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Aug 8 16:33:01 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:04.288442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e402bea-332a-4e29-9d23-b5b1f22b0a43 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:04.289412 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e402bea-332a-4e29-9d23-b5b1f22b0a43 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:04.289842 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e402bea-332a-4e29-9d23-b5b1f22b0a43 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:04.291905 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 486/961] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:33:04.299001 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac24a9-e044-4124-a783-078ca79143b3 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:04.299539 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac24a9-e044-4124-a783-078ca79143b3 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:04.299936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac24a9-e044-4124-a783-078ca79143b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:04.332890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-03ac24a9-e044-4124-a783-078ca79143b3 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:04.335050 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-03ac24a9-e044-4124-a783-078ca79143b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:04.463856 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 476/962] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:33:04.488295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e402bea-332a-4e29-9d23-b5b1f22b0a43 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:04.584322 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5ba49-fdfd-4706-934a-56c5e6e8776b None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:04.584738 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5ba49-fdfd-4706-934a-56c5e6e8776b None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:04.585571 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5ba49-fdfd-4706-934a-56c5e6e8776b None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:04.594934 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 477/963] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:33:04.597945 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb95557-6016-4385-8b1d-a9820d9fe3d2 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:04.598425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb95557-6016-4385-8b1d-a9820d9fe3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:04.598919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb95557-6016-4385-8b1d-a9820d9fe3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:04.633449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:04.645200 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-4fb95557-6016-4385-8b1d-a9820d9fe3d2 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:04.647229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-4fb95557-6016-4385-8b1d-a9820d9fe3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:04.666304 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e402bea-332a-4e29-9d23-b5b1f22b0a43 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:04.711795 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce549800-aa06-4f4b-a762-739846986557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:04.712915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce549800-aa06-4f4b-a762-739846986557 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:04.713410 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce549800-aa06-4f4b-a762-739846986557 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:04.713888 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce549800-aa06-4f4b-a762-739846986557 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:04.728709 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce549800-aa06-4f4b-a762-739846986557 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:33:04.733880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-ce549800-aa06-4f4b-a762-739846986557 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:33:04.733880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:04.733880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:04.733880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:04.733880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:04.733880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:04.733880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:04.733880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:04.733880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:04.733880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:04.733880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:04.733880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:04.733880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:33:04.733880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:33:04.733880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:33:04.733880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:33:04.734882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:33:04.734882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:33:04.734882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:33:04.734882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:33:04.734882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:04.734882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:04.734882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:04.734882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:04.734882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:04.734882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:04.734882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:04.734882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:04.734882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:04.734882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:04.734882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:04.734882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:04.735783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:04.735783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:04.735783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:04.735783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:04.735783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:33:04.735783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:33:04.735783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:33:04.735783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:33:04.735783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:33:04.735783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:33:04.735783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:33:04.735783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:04.743526 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 487/964] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Aug 8 16:33:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:33:04.745317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce549800-aa06-4f4b-a762-739846986557 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:04.767619 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:04.798601 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 478/965] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:33:04.798601 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104611f-5825-4838-a6a6-2360762c2c30 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:04.798601 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104611f-5825-4838-a6a6-2360762c2c30 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:04.798601 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104611f-5825-4838-a6a6-2360762c2c30 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:04.798601 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 479/966] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:33:04.807730 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53961188-2532-405e-87b3-5b0d8d52e10c None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:04.808289 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53961188-2532-405e-87b3-5b0d8d52e10c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:04.808825 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53961188-2532-405e-87b3-5b0d8d52e10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:04.845140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-53961188-2532-405e-87b3-5b0d8d52e10c None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:04.847135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-53961188-2532-405e-87b3-5b0d8d52e10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:04.924200 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce549800-aa06-4f4b-a762-739846986557 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:04.969703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 480/967] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:33:04.970865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53961188-2532-405e-87b3-5b0d8d52e10c None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:04.983610 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503a452-f403-48fb-94bb-0e626fd93b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:04.985026 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503a452-f403-48fb-94bb-0e626fd93b65 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:04.985783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503a452-f403-48fb-94bb-0e626fd93b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:04.986493 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503a452-f403-48fb-94bb-0e626fd93b65 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:04.987924 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f503a452-f403-48fb-94bb-0e626fd93b65 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:04.988706 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f503a452-f403-48fb-94bb-0e626fd93b65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:04.991153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f503a452-f403-48fb-94bb-0e626fd93b65 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:05.008407 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 488/968] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Aug 8 16:33:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:05.016048 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503a452-f403-48fb-94bb-0e626fd93b65 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:05.089132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:05.121449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:05.136091 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53961188-2532-405e-87b3-5b0d8d52e10c None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:05.148796 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503a452-f403-48fb-94bb-0e626fd93b65 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:05.183660 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb1aba-e732-45eb-bbad-05e9e0e1ca02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:05.184332 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb1aba-e732-45eb-bbad-05e9e0e1ca02 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:05.184610 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb1aba-e732-45eb-bbad-05e9e0e1ca02 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:05.184881 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb1aba-e732-45eb-bbad-05e9e0e1ca02 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:05.185490 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb1aba-e732-45eb-bbad-05e9e0e1ca02 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:05.187031 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb1aba-e732-45eb-bbad-05e9e0e1ca02 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:05.189777 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b429d8-e7c1-4375-ab19-cfa8dcfab4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:05.190536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b429d8-e7c1-4375-ab19-cfa8dcfab4a8 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:05.190851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b429d8-e7c1-4375-ab19-cfa8dcfab4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:05.191120 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b429d8-e7c1-4375-ab19-cfa8dcfab4a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:05.197763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-baeb1aba-e732-45eb-bbad-05e9e0e1ca02 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:33:05.197763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:05.197763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:33:05.197763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:33:05.197763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:33:05.197763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:33:05.197763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:33:05.197763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:33:05.197763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:33:05.197763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:05.197763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:33:05.197763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:05.197763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:05.197763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:05.197763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:05.197763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:05.197763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:05.199719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:05.199719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:05.199719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:05.199719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:05.199719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:05.199719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:05.199719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:33:05.199719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:33:05.199719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:33:05.199719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:33:05.199719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:05.199719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:05.199719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:05.199719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:05.199719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:05.199719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:05.201314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:05.201314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:05.201314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:05.201314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:05.201314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:05.201314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:05.201314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:05.201314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:05.201314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:05.201314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:05.201314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:33:05.201314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:33:05.201314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:33:05.201314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:05.201314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 481/969] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Aug 8 16:33:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:05.203058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb1aba-e732-45eb-bbad-05e9e0e1ca02 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:05.210490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b429d8-e7c1-4375-ab19-cfa8dcfab4a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:05.212093 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b429d8-e7c1-4375-ab19-cfa8dcfab4a8 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:05.229063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-39b429d8-e7c1-4375-ab19-cfa8dcfab4a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:33:05.229063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:05.229063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:05.229063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:05.229063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:05.229063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:05.229063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:05.229063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:05.229063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:05.229063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:05.229063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:05.229063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:05.229063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:33:05.229063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:33:05.229063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:33:05.229063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:33:05.231343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:05.231343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:05.231343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:05.231343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:05.231343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:05.231343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:05.231343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:05.231343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:05.231343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:05.231343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:05.231343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:05.231343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:05.231343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:05.231343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:05.231343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:05.231343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:05.231343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:33:05.233382 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:33:05.233382 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:33:05.233382 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:05.233382 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 489/970] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:33:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:33:05.233382 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39bf10-d7cc-4b69-ac7d-7b78c80b1073 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:05.233382 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39bf10-d7cc-4b69-ac7d-7b78c80b1073 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:05.233382 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39bf10-d7cc-4b69-ac7d-7b78c80b1073 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:05.234555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 490/971] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:33:05.234812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a39bf10-d7cc-4b69-ac7d-7b78c80b1073 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:05.346978 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb1aba-e732-45eb-bbad-05e9e0e1ca02 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:05.387722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:05.389021 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a39bf10-d7cc-4b69-ac7d-7b78c80b1073 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:05.396696 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137d3ed-0d7b-49fe-ae52-fba73049615d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:05.403853 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137d3ed-0d7b-49fe-ae52-fba73049615d None admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:05.403853 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137d3ed-0d7b-49fe-ae52-fba73049615d None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:05.403853 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137d3ed-0d7b-49fe-ae52-fba73049615d None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:05.403853 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137d3ed-0d7b-49fe-ae52-fba73049615d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:05.403853 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137d3ed-0d7b-49fe-ae52-fba73049615d None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:05.406452 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: WARNING keystone.server.flask.common [None req-a137d3ed-0d7b-49fe-ae52-fba73049615d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 08 16:33:05.429813 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:05.438007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e41c9-0309-4b42-a55f-dbd2c00ea53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:05.438701 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e41c9-0309-4b42-a55f-dbd2c00ea53d None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:05.439023 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e41c9-0309-4b42-a55f-dbd2c00ea53d None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:05.439349 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e41c9-0309-4b42-a55f-dbd2c00ea53d None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:05.440026 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e41c9-0309-4b42-a55f-dbd2c00ea53d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:05.440364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e41c9-0309-4b42-a55f-dbd2c00ea53d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:05.441854 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e41c9-0309-4b42-a55f-dbd2c00ea53d None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:05.447938 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 491/972] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Aug 8 16:33:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:05.450011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb2d98-05b7-4d39-a72f-cbf4faf01e8c None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:05.450476 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb2d98-05b7-4d39-a72f-cbf4faf01e8c None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:05.450989 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb2d98-05b7-4d39-a72f-cbf4faf01e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:05.452161 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.sql.core [None req-a137d3ed-0d7b-49fe-ae52-fba73049615d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 16:33:05.452161 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 08 16:33:05.452161 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [parameters: {'user_id': '9b70887a056b40379af0a517fec321d2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 08 16:33:05.452161 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81998) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 16:33:05.473078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-a137d3ed-0d7b-49fe-ae52-fba73049615d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 08 16:33:05.473078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:05.473078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 16:33:05.473078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 16:33:05.473078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 16:33:05.473078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 16:33:05.473078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 16:33:05.473078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 16:33:05.473078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 16:33:05.473078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application conn.query(q) Aug 08 16:33:05.473078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 16:33:05.473078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 16:33:05.473078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 16:33:05.473078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application result.read() Aug 08 16:33:05.473078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 16:33:05.474991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 16:33:05.474991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 16:33:05.474991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 16:33:05.474991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 16:33:05.474991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 16:33:05.474991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 16:33:05.474991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 16:33:05.474991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 16:33:05.474991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:05.474991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 16:33:05.474991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:05.474991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:05.474991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 16:33:05.474991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 16:33:05.474991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 08 16:33:05.474991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 08 16:33:05.474991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 16:33:05.474991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application next(self.gen) Aug 08 16:33:05.476914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 16:33:05.476914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application yield resource Aug 08 16:33:05.476914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 16:33:05.476914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application next(self.gen) Aug 08 16:33:05.476914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 16:33:05.476914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 16:33:05.476914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 16:33:05.476914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 16:33:05.476914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 16:33:05.476914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise self.value Aug 08 16:33:05.476914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 16:33:05.476914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 16:33:05.476914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 16:33:05.476914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application session.commit() Aug 08 16:33:05.476914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 16:33:05.476914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 16:33:05.476914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 16:33:05.479809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 16:33:05.479809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 16:33:05.479809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self.session.flush() Aug 08 16:33:05.479809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 16:33:05.479809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 16:33:05.479809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 16:33:05.479809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 16:33:05.479809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 16:33:05.479809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application compat.raise_( Aug 08 16:33:05.479809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 16:33:05.479809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception Aug 08 16:33:05.479809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 16:33:05.479809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 16:33:05.479809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 16:33:05.479809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 16:33:05.479809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 16:33:05.479809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 16:33:05.481460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 16:33:05.481460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 16:33:05.481460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 08 16:33:05.481460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 08 16:33:05.481460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 16:33:05.481460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 16:33:05.481460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 16:33:05.481460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 16:33:05.481460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 16:33:05.481460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 16:33:05.481460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 16:33:05.481460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 16:33:05.481460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 16:33:05.481460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 16:33:05.481460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 16:33:05.481460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception Aug 08 16:33:05.483056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 16:33:05.483056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 16:33:05.483056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 16:33:05.483056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 16:33:05.483056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 16:33:05.483056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 16:33:05.483056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 16:33:05.483056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application conn.query(q) Aug 08 16:33:05.483056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 16:33:05.483056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 16:33:05.483056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 16:33:05.483056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application result.read() Aug 08 16:33:05.483056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 16:33:05.483056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 16:33:05.483056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 16:33:05.483056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 16:33:05.485305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 16:33:05.485305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 16:33:05.485305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 16:33:05.485305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 16:33:05.485305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 16:33:05.485305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 08 16:33:05.485305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [parameters: {'user_id': '9b70887a056b40379af0a517fec321d2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 08 16:33:05.485305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 16:33:05.485305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:05.485305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:33:05.485305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:05.485305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:05.485305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:05.485305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:05.485305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:05.487108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:05.487108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:05.487108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:05.487108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:05.487108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:05.487108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:05.487108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:05.487108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 08 16:33:05.487108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 08 16:33:05.487108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:05.487108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:05.487108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:33:05.487108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:33:05.487108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:33:05.487108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:33:05.487108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 08 16:33:05.487108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 08 16:33:05.490639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 08 16:33:05.490639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 08 16:33:05.490639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 16:33:05.490639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 16:33:05.490639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 08 16:33:05.490639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:05.501953 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-f9cb2d98-05b7-4d39-a72f-cbf4faf01e8c None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:05.501953 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f9cb2d98-05b7-4d39-a72f-cbf4faf01e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:05.502454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 482/973] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Aug 8 16:33:05 2022] POST /identity/v3/users => generated 160 bytes in 276 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:05.502454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137d3ed-0d7b-49fe-ae52-fba73049615d None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:05.580624 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 492/974] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:33:05.582100 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb2d98-05b7-4d39-a72f-cbf4faf01e8c None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:05.623346 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137d3ed-0d7b-49fe-ae52-fba73049615d None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:05.666478 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e1319-27ba-4a31-a103-9d29a8738759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:05.667018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e1319-27ba-4a31-a103-9d29a8738759 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:05.667289 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e1319-27ba-4a31-a103-9d29a8738759 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:05.667554 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e1319-27ba-4a31-a103-9d29a8738759 None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:05.668196 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e1319-27ba-4a31-a103-9d29a8738759 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:05.668488 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e1319-27ba-4a31-a103-9d29a8738759 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:05.670075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e1319-27ba-4a31-a103-9d29a8738759 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:05.686954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 483/975] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:05.688008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e1319-27ba-4a31-a103-9d29a8738759 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:05.732482 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb2d98-05b7-4d39-a72f-cbf4faf01e8c None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:05.762887 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:05.768905 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:05.783942 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d626a-b694-47d6-a1ef-513a4e55513f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:05.784651 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d626a-b694-47d6-a1ef-513a4e55513f None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:05.785088 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d626a-b694-47d6-a1ef-513a4e55513f None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:05.785450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d626a-b694-47d6-a1ef-513a4e55513f None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:05.786428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d626a-b694-47d6-a1ef-513a4e55513f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:05.787099 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d626a-b694-47d6-a1ef-513a4e55513f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:05.788575 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d626a-b694-47d6-a1ef-513a4e55513f None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:05.795406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 493/976] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Aug 8 16:33:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:05.796680 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d626a-b694-47d6-a1ef-513a4e55513f None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:05.833756 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e1319-27ba-4a31-a103-9d29a8738759 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:05.876535 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58eece-a22c-4780-8980-059a3f73548f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:05.877330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58eece-a22c-4780-8980-059a3f73548f None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:05.877683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58eece-a22c-4780-8980-059a3f73548f None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:05.878073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58eece-a22c-4780-8980-059a3f73548f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:05.889384 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58eece-a22c-4780-8980-059a3f73548f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:05.890895 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58eece-a22c-4780-8980-059a3f73548f None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:05.903423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-5c58eece-a22c-4780-8980-059a3f73548f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 16:33:05.903423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:05.903423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:05.903423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:05.903423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:05.903423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:05.903423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:05.903423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:05.903423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:05.903423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:05.903423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:05.903423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:05.903423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:33:05.903423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:33:05.903423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:33:05.903423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:33:05.904531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:05.904531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:05.904531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:33:05.904531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:33:05.904531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:33:05.904531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:33:05.904531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:05.904531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:05.904531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:05.904531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:05.904531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:05.904531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:05.904531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:05.904531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:05.904531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:05.904531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:05.904531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:05.904531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:05.905522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:05.905522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:05.905522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:33:05.905522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:33:05.905522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:33:05.905522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:33:05.905522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:33:05.905522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:33:05.905522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 16:33:05.905522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:05.905522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 484/977] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:33:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:05.906152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58eece-a22c-4780-8980-059a3f73548f None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:05.921788 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d626a-b694-47d6-a1ef-513a4e55513f None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:05.964119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd544334-3dcd-48a2-b937-7583bad629fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:05.965196 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd544334-3dcd-48a2-b937-7583bad629fe None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:05.965698 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd544334-3dcd-48a2-b937-7583bad629fe None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:05.966174 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd544334-3dcd-48a2-b937-7583bad629fe None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:05.967643 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd544334-3dcd-48a2-b937-7583bad629fe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:05.970236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd544334-3dcd-48a2-b937-7583bad629fe None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:06.038116 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58eece-a22c-4780-8980-059a3f73548f None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:06.047177 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 494/978] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Aug 8 16:33:05 2022] POST /identity/v3/services => generated 242 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:33:06.048578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd544334-3dcd-48a2-b937-7583bad629fe None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:06.059278 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:06.098372 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44eddc-2da8-41b6-acc4-ac5a4b775705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:06.099699 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44eddc-2da8-41b6-acc4-ac5a4b775705 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:06.100128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44eddc-2da8-41b6-acc4-ac5a4b775705 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:06.100582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44eddc-2da8-41b6-acc4-ac5a4b775705 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:06.111934 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e44eddc-2da8-41b6-acc4-ac5a4b775705 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:06.113869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e44eddc-2da8-41b6-acc4-ac5a4b775705 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:06.129741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-9e44eddc-2da8-41b6-acc4-ac5a4b775705 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:33:06.129741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:06.129741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:06.129741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:06.129741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:06.129741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:06.129741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:06.129741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:06.129741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:06.129741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:06.129741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:06.129741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:06.129741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:33:06.129741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:33:06.129741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:33:06.129741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:33:06.131765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:06.131765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:06.131765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:33:06.131765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:33:06.131765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:33:06.131765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:33:06.131765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:06.131765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:06.131765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:06.131765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:06.131765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:06.131765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:06.131765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:06.131765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:06.131765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:06.131765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:06.131765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:06.131765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:06.134191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:06.134191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:06.134191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:33:06.134191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:33:06.134191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:33:06.134191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:33:06.134191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:33:06.134191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:33:06.134191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:33:06.134191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:06.134191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 485/979] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Aug 8 16:33:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:06.136135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44eddc-2da8-41b6-acc4-ac5a4b775705 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:06.152467 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:06.246818 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd544334-3dcd-48a2-b937-7583bad629fe None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:06.285629 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44eddc-2da8-41b6-acc4-ac5a4b775705 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:06.299759 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5beb8-95de-4eb1-ad55-a4f97ca7a94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:06.301108 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5beb8-95de-4eb1-ad55-a4f97ca7a94b None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:06.301392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5beb8-95de-4eb1-ad55-a4f97ca7a94b None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:06.301661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5beb8-95de-4eb1-ad55-a4f97ca7a94b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:06.302422 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5beb8-95de-4eb1-ad55-a4f97ca7a94b None admin] RBAC: Adding query filter params (interface=admin, service_id=52005eaa9dce406bafbba660d82e59f0, region_id=RegionOne) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:06.302763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5beb8-95de-4eb1-ad55-a4f97ca7a94b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:06.304203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5beb8-95de-4eb1-ad55-a4f97ca7a94b None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:06.312421 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 495/980] 10.222.0.13 () {56 vars in 1249 bytes} [Mon Aug 8 16:33:06 2022] GET /identity/v3/endpoints?service_id=52005eaa9dce406bafbba660d82e59f0&interface=admin®ion_id=RegionOne => generated 193 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:06.313022 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5beb8-95de-4eb1-ad55-a4f97ca7a94b None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:06.354915 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e94d6-f244-4712-bb1d-4d89a45ea103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:06.355736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e94d6-f244-4712-bb1d-4d89a45ea103 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:06.355878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e94d6-f244-4712-bb1d-4d89a45ea103 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:06.356017 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e94d6-f244-4712-bb1d-4d89a45ea103 None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:06.356745 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e94d6-f244-4712-bb1d-4d89a45ea103 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:06.357144 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e94d6-f244-4712-bb1d-4d89a45ea103 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:06.358533 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e94d6-f244-4712-bb1d-4d89a45ea103 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:06.426615 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 486/981] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Aug 8 16:33:06 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:06.426615 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e94d6-f244-4712-bb1d-4d89a45ea103 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:06.485076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:06.503419 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5beb8-95de-4eb1-ad55-a4f97ca7a94b None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:06.520108 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:06.575177 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f715c76-4028-47c2-bfa6-c2416ac74330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:06.576778 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f715c76-4028-47c2-bfa6-c2416ac74330 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:06.577395 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f715c76-4028-47c2-bfa6-c2416ac74330 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:06.579021 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f715c76-4028-47c2-bfa6-c2416ac74330 None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:06.581081 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f715c76-4028-47c2-bfa6-c2416ac74330 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:06.582028 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f715c76-4028-47c2-bfa6-c2416ac74330 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:06.585685 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f715c76-4028-47c2-bfa6-c2416ac74330 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:06.602149 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e94d6-f244-4712-bb1d-4d89a45ea103 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:06.627471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 496/982] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Aug 8 16:33:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:06.633566 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f715c76-4028-47c2-bfa6-c2416ac74330 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:06.677840 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8d55f-161d-4847-840d-01fac613411c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:06.678889 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8d55f-161d-4847-840d-01fac613411c None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:06.679187 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8d55f-161d-4847-840d-01fac613411c None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:06.679634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8d55f-161d-4847-840d-01fac613411c None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:06.680480 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed8d55f-161d-4847-840d-01fac613411c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:06.683591 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed8d55f-161d-4847-840d-01fac613411c None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:06.703259 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 487/983] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Aug 8 16:33:06 2022] GET /identity/v3/services => generated 1727 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:33:06.767070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f715c76-4028-47c2-bfa6-c2416ac74330 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:06.813041 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39418017-0808-47a4-87bf-c73a83602271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:06.813683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39418017-0808-47a4-87bf-c73a83602271 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:06.813958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39418017-0808-47a4-87bf-c73a83602271 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:06.814301 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39418017-0808-47a4-87bf-c73a83602271 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:06.887448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:06.949616 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39418017-0808-47a4-87bf-c73a83602271 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:06.951178 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39418017-0808-47a4-87bf-c73a83602271 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:07.093530 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 497/984] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:33:06 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:33:07.103486 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8d55f-161d-4847-840d-01fac613411c None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:07.275566 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:07.294955 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8d55f-161d-4847-840d-01fac613411c None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:07.353183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0027d-68dd-4b4c-bfbb-4b7013ba0c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:07.353764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0027d-68dd-4b4c-bfbb-4b7013ba0c45 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:07.354035 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0027d-68dd-4b4c-bfbb-4b7013ba0c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:07.354298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0027d-68dd-4b4c-bfbb-4b7013ba0c45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:07.366425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0027d-68dd-4b4c-bfbb-4b7013ba0c45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:07.368463 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0027d-68dd-4b4c-bfbb-4b7013ba0c45 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:07.387121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-33e0027d-68dd-4b4c-bfbb-4b7013ba0c45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:33:07.387121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:07.387121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:07.387121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:07.387121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:07.387121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:07.387121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:07.387121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:07.387121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:07.387121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:07.387121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:07.387121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:07.387121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:33:07.387121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:33:07.387121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:33:07.387121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:33:07.388257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:07.388257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:07.388257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:07.388257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:07.388257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:07.388257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:07.388257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:07.388257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:07.388257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:07.388257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:07.388257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:07.388257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:07.388257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:07.388257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:07.388257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:07.388257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:07.388257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:33:07.389394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:33:07.389394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:33:07.389394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:07.389394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 488/985] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:33:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:07.397242 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39418017-0808-47a4-87bf-c73a83602271 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:07.482325 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:07.551837 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39418017-0808-47a4-87bf-c73a83602271 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:07.593941 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263a142-4b8d-4e0c-8ac3-afcc3ffec0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:07.594612 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263a142-4b8d-4e0c-8ac3-afcc3ffec0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:07.594862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263a142-4b8d-4e0c-8ac3-afcc3ffec0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:07.595116 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263a142-4b8d-4e0c-8ac3-afcc3ffec0e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:07.595773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263a142-4b8d-4e0c-8ac3-afcc3ffec0e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:07.596066 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263a142-4b8d-4e0c-8ac3-afcc3ffec0e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:07.597465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263a142-4b8d-4e0c-8ac3-afcc3ffec0e5 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:07.612076 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 498/986] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:07.622508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0027d-68dd-4b4c-bfbb-4b7013ba0c45 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:07.745797 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0027d-68dd-4b4c-bfbb-4b7013ba0c45 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:07.786043 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7fa6f0-0b2c-45e9-8870-0eb8c9e91f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:07.786702 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fa6f0-0b2c-45e9-8870-0eb8c9e91f2c None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:07.786975 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fa6f0-0b2c-45e9-8870-0eb8c9e91f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:07.787238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fa6f0-0b2c-45e9-8870-0eb8c9e91f2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:07.795251 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7fa6f0-0b2c-45e9-8870-0eb8c9e91f2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:07.796703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7fa6f0-0b2c-45e9-8870-0eb8c9e91f2c None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:07.809430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-6f7fa6f0-0b2c-45e9-8870-0eb8c9e91f2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:33:07.809430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:07.809430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:07.809430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:07.809430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:07.809430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:07.809430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:07.809430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:07.809430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:07.809430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:07.809430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:07.809430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:07.809430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:33:07.809430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:33:07.809430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:33:07.817917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:33:07.817917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:07.817917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:07.817917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:07.817917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:07.817917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:07.817917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:07.817917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:07.817917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:07.817917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:07.817917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:07.817917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:07.817917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:07.817917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:07.817917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:07.817917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:07.817917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:07.819965 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:33:07.819965 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:33:07.819965 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:33:07.819965 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:33:07.819965 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:33:07.819965 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:33:07.819965 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:33:07.819965 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:07.819965 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 489/987] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:33:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:33:07.821437 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263a142-4b8d-4e0c-8ac3-afcc3ffec0e5 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:07.977293 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263a142-4b8d-4e0c-8ac3-afcc3ffec0e5 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:08.027306 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:08.041491 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde5848-adf2-46dc-871e-47843d61e790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:08.044354 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde5848-adf2-46dc-871e-47843d61e790 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:08.044354 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde5848-adf2-46dc-871e-47843d61e790 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:08.044354 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde5848-adf2-46dc-871e-47843d61e790 None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:08.044801 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde5848-adf2-46dc-871e-47843d61e790 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:08.045286 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde5848-adf2-46dc-871e-47843d61e790 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:08.048072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde5848-adf2-46dc-871e-47843d61e790 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:08.078179 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 499/988] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:33:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:08.079880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7fa6f0-0b2c-45e9-8870-0eb8c9e91f2c None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:08.084134 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:08.228818 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7fa6f0-0b2c-45e9-8870-0eb8c9e91f2c None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:08.285187 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a46a0-6390-4526-b2fb-33c77d1cd5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:08.286373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a46a0-6390-4526-b2fb-33c77d1cd5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:08.286989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a46a0-6390-4526-b2fb-33c77d1cd5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:08.287522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a46a0-6390-4526-b2fb-33c77d1cd5c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:08.289740 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a46a0-6390-4526-b2fb-33c77d1cd5c5 None admin] RBAC: Adding query filter params (role.id=16940dcad5d247d0b43da26eb2b69cbc, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=9646662b0b3048f79ad5d2b9602f10a6) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:08.290280 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a46a0-6390-4526-b2fb-33c77d1cd5c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:08.293054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a46a0-6390-4526-b2fb-33c77d1cd5c5 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:08.310622 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 490/989] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:33:08 2022] GET /identity/v3/role_assignments?user.id=9646662b0b3048f79ad5d2b9602f10a6&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=16940dcad5d247d0b43da26eb2b69cbc => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:08.632046 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996248ba-b6ef-4630-ad73-e18d88d609d9 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:08.633157 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996248ba-b6ef-4630-ad73-e18d88d609d9 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:08.633523 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996248ba-b6ef-4630-ad73-e18d88d609d9 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:08.636344 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 500/990] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:33:08.642328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4b809-4d3d-44aa-b260-1cd95cef8e47 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:08.643169 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4b809-4d3d-44aa-b260-1cd95cef8e47 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:08.643169 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4b809-4d3d-44aa-b260-1cd95cef8e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:08.664707 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-fdb4b809-4d3d-44aa-b260-1cd95cef8e47 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:08.666330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-fdb4b809-4d3d-44aa-b260-1cd95cef8e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:08.741099 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:08.750840 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 491/991] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:08 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:33:08.775994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996248ba-b6ef-4630-ad73-e18d88d609d9 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:08.955360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996248ba-b6ef-4630-ad73-e18d88d609d9 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:08.999894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec03bf-63c4-4385-b9fd-13875bf7eed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:09.000555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec03bf-63c4-4385-b9fd-13875bf7eed1 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:09.000827 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec03bf-63c4-4385-b9fd-13875bf7eed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:09.001093 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec03bf-63c4-4385-b9fd-13875bf7eed1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:09.001694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec03bf-63c4-4385-b9fd-13875bf7eed1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:09.003145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec03bf-63c4-4385-b9fd-13875bf7eed1 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:09.003757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87663ec4-2ce2-4e05-8aa0-4b9e3a495783 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:09.004586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87663ec4-2ce2-4e05-8aa0-4b9e3a495783 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:09.005713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87663ec4-2ce2-4e05-8aa0-4b9e3a495783 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:09.010077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 492/992] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:08 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:33:09.013785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-49ec03bf-63c4-4385-b9fd-13875bf7eed1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 16:33:09.013785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:09.013785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:33:09.013785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:33:09.013785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:33:09.013785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:33:09.013785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:33:09.013785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:33:09.013785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 16:33:09.013785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:09.013785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:33:09.013785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:09.013785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:09.013785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:09.013785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:09.013785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:09.013785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:09.015974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:09.015974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:09.015974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:09.015974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:09.015974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:09.015974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:09.015974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:33:09.015974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:33:09.015974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:33:09.015974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:33:09.015974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:09.015974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:09.015974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:09.015974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:09.015974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:09.015974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:09.017901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:09.017901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:09.017901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:09.017901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:09.017901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:09.017901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:09.017901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:09.017901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:09.017901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:09.017901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:09.017901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:33:09.017901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:33:09.017901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 16:33:09.017901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:09.017901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 501/993] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:33:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:09.021203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec03bf-63c4-4385-b9fd-13875bf7eed1 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:09.023018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef1edb-823f-44ad-a6b1-2d947488b800 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:09.023993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef1edb-823f-44ad-a6b1-2d947488b800 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:09.024452 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef1edb-823f-44ad-a6b1-2d947488b800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:09.061111 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-20ef1edb-823f-44ad-a6b1-2d947488b800 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:09.063129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-20ef1edb-823f-44ad-a6b1-2d947488b800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:09.084055 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:09.150635 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 493/994] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:09 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:33:09.171898 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef1edb-823f-44ad-a6b1-2d947488b800 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:09.179164 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec03bf-63c4-4385-b9fd-13875bf7eed1 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:09.229268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204273a-98d3-4da7-be71-e929ffbf1c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:09.230430 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204273a-98d3-4da7-be71-e929ffbf1c76 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:09.230897 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204273a-98d3-4da7-be71-e929ffbf1c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:09.231335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204273a-98d3-4da7-be71-e929ffbf1c76 None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:09.232390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204273a-98d3-4da7-be71-e929ffbf1c76 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:09.232883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204273a-98d3-4da7-be71-e929ffbf1c76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:09.235066 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204273a-98d3-4da7-be71-e929ffbf1c76 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:09.243823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 502/995] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Aug 8 16:33:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:09.253126 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204273a-98d3-4da7-be71-e929ffbf1c76 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:09.328868 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef1edb-823f-44ad-a6b1-2d947488b800 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:09.376613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:09.390507 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf286aba-82b2-4672-818d-44d8e4cd31e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:09.391526 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf286aba-82b2-4672-818d-44d8e4cd31e6 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:09.391972 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf286aba-82b2-4672-818d-44d8e4cd31e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:09.392409 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf286aba-82b2-4672-818d-44d8e4cd31e6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:09.393384 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf286aba-82b2-4672-818d-44d8e4cd31e6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:09.395516 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf286aba-82b2-4672-818d-44d8e4cd31e6 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:09.412015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-cf286aba-82b2-4672-818d-44d8e4cd31e6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:33:09.412015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:09.412015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:33:09.412015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:33:09.412015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:33:09.412015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:33:09.412015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:33:09.412015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:33:09.412015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:33:09.412015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:09.412015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:33:09.412015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:09.412015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:09.412015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:09.412015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:09.412015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:09.412015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:09.413315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:09.413315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:09.413315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:09.413315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:09.413315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:09.413315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:09.413315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:33:09.413315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:33:09.413315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:33:09.413315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:33:09.413315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:09.413315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:09.413315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:09.413315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:09.413315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:09.413315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:09.414894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:09.414894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:09.414894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:09.414894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:09.414894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:09.414894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:09.414894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:09.414894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:09.414894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:09.414894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:09.414894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:33:09.414894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:33:09.414894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:33:09.414894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:09.416932 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204273a-98d3-4da7-be71-e929ffbf1c76 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:09.423399 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 494/996] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Aug 8 16:33:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:33:09.424850 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf286aba-82b2-4672-818d-44d8e4cd31e6 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:09.437369 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:09.463388 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ca080-0db5-46c7-b3d5-b4287da419fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:09.464490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ca080-0db5-46c7-b3d5-b4287da419fc None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:09.464982 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ca080-0db5-46c7-b3d5-b4287da419fc None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:09.465490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ca080-0db5-46c7-b3d5-b4287da419fc None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:09.466723 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ca080-0db5-46c7-b3d5-b4287da419fc None admin] RBAC: Adding query filter params (type=compute) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:09.467485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ca080-0db5-46c7-b3d5-b4287da419fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:09.470150 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ca080-0db5-46c7-b3d5-b4287da419fc None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:09.510680 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 503/997] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Aug 8 16:33:09 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:09.518393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ca080-0db5-46c7-b3d5-b4287da419fc None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:09.568762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf286aba-82b2-4672-818d-44d8e4cd31e6 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:09.624540 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f556e4-b8e9-4efa-9502-930b979a7ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:09.625369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f556e4-b8e9-4efa-9502-930b979a7ddf None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:09.625755 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f556e4-b8e9-4efa-9502-930b979a7ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:09.626134 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f556e4-b8e9-4efa-9502-930b979a7ddf None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:09.627192 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f556e4-b8e9-4efa-9502-930b979a7ddf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:09.628844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f556e4-b8e9-4efa-9502-930b979a7ddf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:09.629757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f556e4-b8e9-4efa-9502-930b979a7ddf None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:09.641878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 495/998] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Aug 8 16:33:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:09.642943 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f556e4-b8e9-4efa-9502-930b979a7ddf None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:09.684921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ca080-0db5-46c7-b3d5-b4287da419fc None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:09.720181 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:09.733495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fd881-25cc-4c0c-a75e-99520c778608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:09.734469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fd881-25cc-4c0c-a75e-99520c778608 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:09.734959 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fd881-25cc-4c0c-a75e-99520c778608 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:09.735334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fd881-25cc-4c0c-a75e-99520c778608 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:09.736165 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0fd881-25cc-4c0c-a75e-99520c778608 None admin] RBAC: Adding query filter params (interface=public, service_id=250ba64c7acd41d4b4de002882f9a839, region_id=RegionOne) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:09.736522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0fd881-25cc-4c0c-a75e-99520c778608 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:09.738202 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0fd881-25cc-4c0c-a75e-99520c778608 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:09.745748 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 504/999] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Aug 8 16:33:09 2022] GET /identity/v3/endpoints?service_id=250ba64c7acd41d4b4de002882f9a839&interface=public®ion_id=RegionOne => generated 194 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:09.752824 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fd881-25cc-4c0c-a75e-99520c778608 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:09.781020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f556e4-b8e9-4efa-9502-930b979a7ddf None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:09.813226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:09.828569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64acbe-dfd0-4beb-b0ed-4f01e64a005e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:09.830045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64acbe-dfd0-4beb-b0ed-4f01e64a005e None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:09.830833 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64acbe-dfd0-4beb-b0ed-4f01e64a005e None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:09.831613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64acbe-dfd0-4beb-b0ed-4f01e64a005e None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:09.832854 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64acbe-dfd0-4beb-b0ed-4f01e64a005e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:09.833387 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64acbe-dfd0-4beb-b0ed-4f01e64a005e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:09.836153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64acbe-dfd0-4beb-b0ed-4f01e64a005e None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:09.846074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 496/1000] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Aug 8 16:33:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:09.853561 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64acbe-dfd0-4beb-b0ed-4f01e64a005e None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:09.906540 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fd881-25cc-4c0c-a75e-99520c778608 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:09.965320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef299c-1353-4471-a9f0-dccb22e14087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:09.966160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef299c-1353-4471-a9f0-dccb22e14087 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:09.966532 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef299c-1353-4471-a9f0-dccb22e14087 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:09.967015 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef299c-1353-4471-a9f0-dccb22e14087 None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:09.968020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef299c-1353-4471-a9f0-dccb22e14087 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:09.969521 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef299c-1353-4471-a9f0-dccb22e14087 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:09.978184 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 505/1001] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Aug 8 16:33:09 2022] GET /identity/v3/services => generated 1727 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:33:10.018460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64acbe-dfd0-4beb-b0ed-4f01e64a005e None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:10.041084 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:10.063892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c03f3-f030-4e3f-9329-d24f40fff26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:10.064806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c03f3-f030-4e3f-9329-d24f40fff26b None admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:10.065227 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c03f3-f030-4e3f-9329-d24f40fff26b None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:10.065623 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c03f3-f030-4e3f-9329-d24f40fff26b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:10.066492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c03f3-f030-4e3f-9329-d24f40fff26b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:10.068101 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c03f3-f030-4e3f-9329-d24f40fff26b None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:10.156494 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 497/1002] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:33:09 2022] POST /identity/v3/endpoints => generated 325 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:33:10.753098 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02be5f-d649-470f-93f6-2fa5cf765749 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:10.753953 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02be5f-d649-470f-93f6-2fa5cf765749 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:10.754088 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02be5f-d649-470f-93f6-2fa5cf765749 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:10.756775 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 506/1003] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:33:10.765804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7aa04-f444-4029-b40b-58eaacd1209c None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:10.766612 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7aa04-f444-4029-b40b-58eaacd1209c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:10.766770 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7aa04-f444-4029-b40b-58eaacd1209c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:10.798443 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-7ce7aa04-f444-4029-b40b-58eaacd1209c None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:10.800417 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-7ce7aa04-f444-4029-b40b-58eaacd1209c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:10.938613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 498/1004] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:10 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:33:10.962178 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b02be5f-d649-470f-93f6-2fa5cf765749 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:11.086611 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b02be5f-d649-470f-93f6-2fa5cf765749 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:11.098286 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:11.128086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25219b5c-2449-4995-96d8-a6a1b5137a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:11.128732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25219b5c-2449-4995-96d8-a6a1b5137a71 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:11.129026 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25219b5c-2449-4995-96d8-a6a1b5137a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:11.129304 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25219b5c-2449-4995-96d8-a6a1b5137a71 None admin] PATH_INFO: `/v3/services/image` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:11.129854 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25219b5c-2449-4995-96d8-a6a1b5137a71 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:11.131368 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25219b5c-2449-4995-96d8-a6a1b5137a71 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:11.141921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-25219b5c-2449-4995-96d8-a6a1b5137a71 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 16:33:11.141921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:11.141921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:33:11.141921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:33:11.141921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:33:11.141921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:33:11.141921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:33:11.141921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:33:11.141921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 16:33:11.141921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:11.141921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:33:11.141921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:11.141921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:11.141921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:11.141921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:11.141921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:11.141921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:11.149892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:11.149892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:11.149892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:11.149892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:11.149892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:11.149892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:11.149892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:33:11.149892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:33:11.149892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:33:11.149892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:33:11.149892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:11.149892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:11.149892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:11.149892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:11.149892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:11.149892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:11.151495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:11.151495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:11.151495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:11.151495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:11.151495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:11.151495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:11.151495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:11.151495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:11.151495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:11.151495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:11.151495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:33:11.151495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:33:11.151495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 16:33:11.151495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:11.151495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 507/1005] 10.222.0.13 () {56 vars in 1105 bytes} [Mon Aug 8 16:33:10 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:11.158667 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce7aa04-f444-4029-b40b-58eaacd1209c None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:11.337000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce7aa04-f444-4029-b40b-58eaacd1209c None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:11.341252 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:11.400039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acedfab-98fa-407f-b6da-0381c3ecafd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:11.401075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acedfab-98fa-407f-b6da-0381c3ecafd9 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:11.401563 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acedfab-98fa-407f-b6da-0381c3ecafd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:11.402032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acedfab-98fa-407f-b6da-0381c3ecafd9 None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:11.403199 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acedfab-98fa-407f-b6da-0381c3ecafd9 None admin] RBAC: Adding query filter params (name=image) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:11.403721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acedfab-98fa-407f-b6da-0381c3ecafd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:11.407118 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acedfab-98fa-407f-b6da-0381c3ecafd9 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:11.416512 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 499/1006] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:33:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:11.425512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25219b5c-2449-4995-96d8-a6a1b5137a71 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:11.558443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25219b5c-2449-4995-96d8-a6a1b5137a71 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:11.598498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb4234-8816-4d96-afc6-520583787419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:11.599034 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb4234-8816-4d96-afc6-520583787419 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:11.599311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb4234-8816-4d96-afc6-520583787419 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:11.599584 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb4234-8816-4d96-afc6-520583787419 None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:11.600228 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb4234-8816-4d96-afc6-520583787419 None admin] RBAC: Adding query filter params (type=image) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:11.600522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb4234-8816-4d96-afc6-520583787419 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:11.602067 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb4234-8816-4d96-afc6-520583787419 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:11.612977 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 508/1007] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:33:11 2022] GET /identity/v3/services?type=image => generated 122 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:12.443947 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac33c7-09a9-452b-919d-efea4cb981d7 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:12.444780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac33c7-09a9-452b-919d-efea4cb981d7 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:12.444780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac33c7-09a9-452b-919d-efea4cb981d7 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:12.448142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 500/1008] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:33:12.455319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5670797-904f-427a-8a7b-b6b59fcc3599 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:12.455949 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5670797-904f-427a-8a7b-b6b59fcc3599 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:12.456382 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5670797-904f-427a-8a7b-b6b59fcc3599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:12.486942 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-a5670797-904f-427a-8a7b-b6b59fcc3599 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:12.489137 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-a5670797-904f-427a-8a7b-b6b59fcc3599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:12.543473 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:12.594636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 509/1009] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:12 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:33:12.618773 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac33c7-09a9-452b-919d-efea4cb981d7 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:12.743961 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac33c7-09a9-452b-919d-efea4cb981d7 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:12.784896 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93770b5b-fe27-4830-b0e4-3f5565bf4c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:12.785537 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93770b5b-fe27-4830-b0e4-3f5565bf4c49 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:12.785811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93770b5b-fe27-4830-b0e4-3f5565bf4c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:12.786076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93770b5b-fe27-4830-b0e4-3f5565bf4c49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:12.787303 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93770b5b-fe27-4830-b0e4-3f5565bf4c49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:12.788856 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93770b5b-fe27-4830-b0e4-3f5565bf4c49 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:12.790910 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 501/1010] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:33:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:12.797339 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5670797-904f-427a-8a7b-b6b59fcc3599 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:12.938727 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5670797-904f-427a-8a7b-b6b59fcc3599 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:12.980962 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f0c04-61be-487b-aeae-dfeaa00b7f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:12.981778 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f0c04-61be-487b-aeae-dfeaa00b7f6e None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:12.982180 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f0c04-61be-487b-aeae-dfeaa00b7f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:12.982594 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f0c04-61be-487b-aeae-dfeaa00b7f6e None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:12.983405 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f0c04-61be-487b-aeae-dfeaa00b7f6e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:12.985054 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f0c04-61be-487b-aeae-dfeaa00b7f6e None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:13.186819 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 510/1011] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Aug 8 16:33:12 2022] POST /identity/v3/projects => generated 311 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:33:13.530117 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9da67-0b64-408f-ad6b-651a1c619010 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:13.530867 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9da67-0b64-408f-ad6b-651a1c619010 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:13.531003 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9da67-0b64-408f-ad6b-651a1c619010 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:13.533781 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 502/1012] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:33:13.543605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd03c8-b44a-485f-8b62-b721d3f42e2f None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:13.544304 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd03c8-b44a-485f-8b62-b721d3f42e2f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:13.544599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd03c8-b44a-485f-8b62-b721d3f42e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:13.565766 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-b6bd03c8-b44a-485f-8b62-b721d3f42e2f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:13.567208 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b6bd03c8-b44a-485f-8b62-b721d3f42e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:13.584407 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:13.689596 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 511/1013] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:13 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:33:13.714934 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e9da67-0b64-408f-ad6b-651a1c619010 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:13.764992 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:13.825452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10faf4b4-f18f-4c14-88b6-363beba81a8b None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:13.825654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10faf4b4-f18f-4c14-88b6-363beba81a8b None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:13.825913 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10faf4b4-f18f-4c14-88b6-363beba81a8b None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:13.827565 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 512/1014] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:33:13.833143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a09b3a-816c-48f2-8ebe-ed3a0a8338d1 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:13.833413 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a09b3a-816c-48f2-8ebe-ed3a0a8338d1 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:13.833683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a09b3a-816c-48f2-8ebe-ed3a0a8338d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:13.847542 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e9da67-0b64-408f-ad6b-651a1c619010 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:13.854099 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-17a09b3a-816c-48f2-8ebe-ed3a0a8338d1 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:13.855452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-17a09b3a-816c-48f2-8ebe-ed3a0a8338d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:13.893713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbc878-8f48-439e-8686-740a08aaa5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:13.894474 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbc878-8f48-439e-8686-740a08aaa5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:13.894804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbc878-8f48-439e-8686-740a08aaa5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:13.895078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbc878-8f48-439e-8686-740a08aaa5a6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:13.895803 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbc878-8f48-439e-8686-740a08aaa5a6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:13.897349 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbc878-8f48-439e-8686-740a08aaa5a6 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:13.909318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-27bbc878-8f48-439e-8686-740a08aaa5a6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 16:33:13.909318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:13.909318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:33:13.909318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:33:13.909318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:33:13.909318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:33:13.909318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:33:13.909318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:33:13.909318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 16:33:13.909318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:13.909318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:33:13.909318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:13.909318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:13.909318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:13.909318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:13.909318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:13.909318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:13.910518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:13.910518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:13.910518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:13.910518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:13.910518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:13.910518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:13.910518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:33:13.910518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:33:13.910518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:33:13.910518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:33:13.910518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:13.910518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:13.910518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:13.910518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:13.910518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:13.910518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:13.911673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:13.911673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:13.911673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:13.911673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:13.911673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:13.911673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:13.911673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:13.911673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:13.911673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:13.911673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:13.911673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:33:13.911673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:33:13.911673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 16:33:13.911673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:13.911673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 503/1015] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:33:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:13.918591 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbc878-8f48-439e-8686-740a08aaa5a6 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:13.961127 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 513/1016] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:13 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:33:13.985842 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a09b3a-816c-48f2-8ebe-ed3a0a8338d1 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:14.098163 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:14.099646 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:14.106080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbc878-8f48-439e-8686-740a08aaa5a6 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:14.125465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a09b3a-816c-48f2-8ebe-ed3a0a8338d1 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:14.167656 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941d307-d7d1-4215-9fa0-556eb3fd2025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:14.168290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941d307-d7d1-4215-9fa0-556eb3fd2025 None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:14.168566 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941d307-d7d1-4215-9fa0-556eb3fd2025 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:14.168845 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941d307-d7d1-4215-9fa0-556eb3fd2025 None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:14.169620 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941d307-d7d1-4215-9fa0-556eb3fd2025 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:14.169786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357f9d0-95ac-4780-bd7b-8ac8792a86a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:14.170422 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357f9d0-95ac-4780-bd7b-8ac8792a86a7 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:14.170729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357f9d0-95ac-4780-bd7b-8ac8792a86a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:14.171000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357f9d0-95ac-4780-bd7b-8ac8792a86a7 None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:14.171178 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941d307-d7d1-4215-9fa0-556eb3fd2025 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:14.171681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357f9d0-95ac-4780-bd7b-8ac8792a86a7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:14.172125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357f9d0-95ac-4780-bd7b-8ac8792a86a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:14.174734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357f9d0-95ac-4780-bd7b-8ac8792a86a7 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:14.184477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 504/1017] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Aug 8 16:33:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:14.191304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357f9d0-95ac-4780-bd7b-8ac8792a86a7 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:14.250639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 514/1018] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Aug 8 16:33:13 2022] POST /identity/v3/services => generated 242 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:33:14.370505 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357f9d0-95ac-4780-bd7b-8ac8792a86a7 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:14.422559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136261a3-7fdb-439e-a94e-d8ebb01072e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:14.423239 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136261a3-7fdb-439e-a94e-d8ebb01072e4 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:14.423510 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136261a3-7fdb-439e-a94e-d8ebb01072e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:14.423774 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136261a3-7fdb-439e-a94e-d8ebb01072e4 None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:14.424433 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136261a3-7fdb-439e-a94e-d8ebb01072e4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:14.424732 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136261a3-7fdb-439e-a94e-d8ebb01072e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:14.426187 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136261a3-7fdb-439e-a94e-d8ebb01072e4 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:14.433191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 505/1019] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Aug 8 16:33:14 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:14.440281 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941d307-d7d1-4215-9fa0-556eb3fd2025 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:14.610492 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941d307-d7d1-4215-9fa0-556eb3fd2025 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:14.686195 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07771b-dcc5-437b-ac84-a9311afb660d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:14.686862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07771b-dcc5-437b-ac84-a9311afb660d None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:14.687239 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07771b-dcc5-437b-ac84-a9311afb660d None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:14.687575 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07771b-dcc5-437b-ac84-a9311afb660d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:14.688299 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb07771b-dcc5-437b-ac84-a9311afb660d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:14.689955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb07771b-dcc5-437b-ac84-a9311afb660d None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:14.705943 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:14.790674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 515/1020] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:33:14 2022] POST /identity/v3/endpoints => generated 334 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:33:15.417686 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa81c24-ab07-4c99-a552-c56d6932b417 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:15.418549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa81c24-ab07-4c99-a552-c56d6932b417 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:15.418549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa81c24-ab07-4c99-a552-c56d6932b417 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:15.419853 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 506/1021] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:33:15.425374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d16ef-a558-4075-84cb-0bc065b29cac None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:15.425661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d16ef-a558-4075-84cb-0bc065b29cac None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:15.425960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d16ef-a558-4075-84cb-0bc065b29cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:15.447185 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-a98d16ef-a558-4075-84cb-0bc065b29cac None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:15.448773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-a98d16ef-a558-4075-84cb-0bc065b29cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:15.545997 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 516/1022] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:33:15.571171 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa81c24-ab07-4c99-a552-c56d6932b417 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:15.611617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:15.704611 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa81c24-ab07-4c99-a552-c56d6932b417 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:15.745822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b666b-7d11-4756-918e-b5ed026c1803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:15.746698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b666b-7d11-4756-918e-b5ed026c1803 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:15.747118 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b666b-7d11-4756-918e-b5ed026c1803 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:15.747573 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b666b-7d11-4756-918e-b5ed026c1803 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:15.748914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b666b-7d11-4756-918e-b5ed026c1803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:15.750632 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b666b-7d11-4756-918e-b5ed026c1803 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:15.752664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 507/1023] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:33:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:15.759641 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d16ef-a558-4075-84cb-0bc065b29cac None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:15.886411 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d16ef-a558-4075-84cb-0bc065b29cac None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:15.912201 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:15.942333 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c942411-d65a-414f-93f9-a87608ffeedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:15.943201 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c942411-d65a-414f-93f9-a87608ffeedf None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:15.943619 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c942411-d65a-414f-93f9-a87608ffeedf None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:15.944048 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c942411-d65a-414f-93f9-a87608ffeedf None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:15.945447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c942411-d65a-414f-93f9-a87608ffeedf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:15.947444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c942411-d65a-414f-93f9-a87608ffeedf None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:16.065543 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 517/1024] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Aug 8 16:33:15 2022] POST /identity/v3/users => generated 285 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:33:16.477459 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3306ebf-10b3-4ca4-b803-dc97f3fd0713 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:16.478421 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3306ebf-10b3-4ca4-b803-dc97f3fd0713 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:16.478421 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3306ebf-10b3-4ca4-b803-dc97f3fd0713 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:16.482609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 508/1025] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:33:16.488582 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0473c7d-782e-4e6b-8aa2-48d62251b453 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:16.492396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0473c7d-782e-4e6b-8aa2-48d62251b453 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:16.492396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0473c7d-782e-4e6b-8aa2-48d62251b453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:16.523004 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-f0473c7d-782e-4e6b-8aa2-48d62251b453 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:16.525220 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f0473c7d-782e-4e6b-8aa2-48d62251b453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:16.654284 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 518/1026] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:33:16.696590 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3306ebf-10b3-4ca4-b803-dc97f3fd0713 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:16.887343 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:16.907223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3306ebf-10b3-4ca4-b803-dc97f3fd0713 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:16.975142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb6eeb-f404-4cb1-b03c-7a63c18a451d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:16.976773 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb6eeb-f404-4cb1-b03c-7a63c18a451d None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:16.977249 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb6eeb-f404-4cb1-b03c-7a63c18a451d None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:16.977712 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb6eeb-f404-4cb1-b03c-7a63c18a451d None admin] PATH_INFO: `/v3/services/image` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:16.978752 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb6eeb-f404-4cb1-b03c-7a63c18a451d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:16.981229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb6eeb-f404-4cb1-b03c-7a63c18a451d None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:16.998447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-70cb6eeb-f404-4cb1-b03c-7a63c18a451d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 16:33:16.998447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:16.998447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:33:16.998447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:33:16.998447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:33:16.998447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:33:16.998447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:33:16.998447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:33:16.998447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 16:33:16.998447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:16.998447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:33:16.998447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:16.998447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:16.998447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:16.998447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:16.998447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:16.998447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:17.000548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:17.000548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:17.000548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:17.000548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:17.000548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:17.000548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:17.000548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:33:17.000548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:33:17.000548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:33:17.000548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:33:17.000548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:17.000548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:17.000548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:17.000548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:17.000548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:17.000548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:17.002304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:17.002304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:17.002304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:17.002304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:17.002304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:17.002304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:17.002304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:17.002304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:17.002304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:17.002304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:17.002304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:33:17.002304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:33:17.002304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 16:33:17.002304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:17.002304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 509/1027] 10.222.0.13 () {56 vars in 1105 bytes} [Mon Aug 8 16:33:16 2022] GET /identity/v3/services/image => generated 86 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:17.014067 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0473c7d-782e-4e6b-8aa2-48d62251b453 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:17.211533 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0473c7d-782e-4e6b-8aa2-48d62251b453 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:17.226858 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:17.280673 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5049535-4c43-4960-afdf-19aae65df6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:17.282064 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5049535-4c43-4960-afdf-19aae65df6b9 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:17.282741 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5049535-4c43-4960-afdf-19aae65df6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:17.283444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5049535-4c43-4960-afdf-19aae65df6b9 None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:17.284952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5049535-4c43-4960-afdf-19aae65df6b9 None admin] RBAC: Adding query filter params (name=image) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:17.285715 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5049535-4c43-4960-afdf-19aae65df6b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:17.288532 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5049535-4c43-4960-afdf-19aae65df6b9 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:17.304613 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 519/1028] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:33:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:17.309774 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb6eeb-f404-4cb1-b03c-7a63c18a451d None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:17.512106 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb6eeb-f404-4cb1-b03c-7a63c18a451d None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:17.578164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227f424-374b-4644-9d3c-a22b911eef86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:17.579645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227f424-374b-4644-9d3c-a22b911eef86 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:17.580406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227f424-374b-4644-9d3c-a22b911eef86 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:17.581146 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227f424-374b-4644-9d3c-a22b911eef86 None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:17.582549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227f424-374b-4644-9d3c-a22b911eef86 None admin] RBAC: Adding query filter params (type=image) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:17.583351 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227f424-374b-4644-9d3c-a22b911eef86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:17.586797 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227f424-374b-4644-9d3c-a22b911eef86 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:17.602658 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 510/1029] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:33:17 2022] GET /identity/v3/services?type=image => generated 350 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:17.608144 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5049535-4c43-4960-afdf-19aae65df6b9 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:17.774954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb30fe9-9686-47d9-91c1-87f40835f1c9 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:17.775563 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb30fe9-9686-47d9-91c1-87f40835f1c9 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:17.776040 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb30fe9-9686-47d9-91c1-87f40835f1c9 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:17.778687 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 511/1030] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:33:17.784828 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b4de0-cf47-48f2-9931-644e729460d3 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:17.785104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b4de0-cf47-48f2-9931-644e729460d3 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:17.785370 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b4de0-cf47-48f2-9931-644e729460d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:17.805753 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-ed6b4de0-cf47-48f2-9931-644e729460d3 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:17.807037 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ed6b4de0-cf47-48f2-9931-644e729460d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:17.807460 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5049535-4c43-4960-afdf-19aae65df6b9 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:17.824287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:17.879325 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4febec-847e-487b-889d-0384062bfe01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:17.880131 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4febec-847e-487b-889d-0384062bfe01 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:17.880541 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4febec-847e-487b-889d-0384062bfe01 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:17.881629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4febec-847e-487b-889d-0384062bfe01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:17.882172 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4febec-847e-487b-889d-0384062bfe01 None admin] RBAC: Adding query filter params (interface=public, service_id=986f1c2a8c1143fa8c62b12cedb511c8, region_id=RegionOne) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:17.882860 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4febec-847e-487b-889d-0384062bfe01 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:17.884383 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4febec-847e-487b-889d-0384062bfe01 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:17.892861 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 520/1031] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Aug 8 16:33:17 2022] GET /identity/v3/endpoints?service_id=986f1c2a8c1143fa8c62b12cedb511c8&interface=public®ion_id=RegionOne => generated 194 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:17.900912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4febec-847e-487b-889d-0384062bfe01 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:17.909293 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 512/1032] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:33:17.940435 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b4de0-cf47-48f2-9931-644e729460d3 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:17.975485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:18.101740 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4febec-847e-487b-889d-0384062bfe01 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:18.114778 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b4de0-cf47-48f2-9931-644e729460d3 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:18.161193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ba222-7473-40f8-b759-6e75874eee6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:18.162239 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ba222-7473-40f8-b759-6e75874eee6a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:18.162644 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ba222-7473-40f8-b759-6e75874eee6a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:18.163109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ba222-7473-40f8-b759-6e75874eee6a None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:18.164214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ba222-7473-40f8-b759-6e75874eee6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:18.166631 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ba222-7473-40f8-b759-6e75874eee6a None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:18.177550 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f0ee8-da03-4817-913b-01941a6411a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:18.178659 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f0ee8-da03-4817-913b-01941a6411a5 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:18.179171 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f0ee8-da03-4817-913b-01941a6411a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:18.179796 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f0ee8-da03-4817-913b-01941a6411a5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:18.190873 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 521/1033] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Aug 8 16:33:17 2022] GET /identity/v3/services => generated 1957 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:33:18.211895 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f0ee8-da03-4817-913b-01941a6411a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:33:18.221488 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-6c2f0ee8-da03-4817-913b-01941a6411a5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:33:18.221488 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:18.221488 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:18.221488 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:18.221488 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:18.221488 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:18.221488 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:18.221488 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:18.221488 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:18.221488 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:18.221488 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:18.221488 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:18.221488 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:33:18.221488 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:33:18.221488 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:33:18.221488 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:33:18.222737 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:33:18.222737 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:33:18.222737 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:33:18.222737 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:33:18.222737 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:18.222737 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:18.222737 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:18.222737 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:18.222737 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:18.222737 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:18.222737 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:18.222737 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:18.222737 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:18.222737 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:18.222737 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:18.222737 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:18.223798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:18.223798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:18.223798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:18.223798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:18.223798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:33:18.223798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:33:18.223798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:33:18.223798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:33:18.223798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:33:18.223798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:33:18.223798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:33:18.223798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:18.224655 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 513/1034] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Aug 8 16:33:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:33:18.232689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ba222-7473-40f8-b759-6e75874eee6a None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:18.425023 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:18.429050 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfe574-d430-4adf-a67d-cd1ee6c11b52 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:18.430010 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ba222-7473-40f8-b759-6e75874eee6a None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:18.432255 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfe574-d430-4adf-a67d-cd1ee6c11b52 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:18.432684 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfe574-d430-4adf-a67d-cd1ee6c11b52 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:18.436089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 514/1035] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:33:18.444298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ea347-d28e-478d-9e5c-37072f01559c None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:18.445463 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ea347-d28e-478d-9e5c-37072f01559c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:18.446030 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ea347-d28e-478d-9e5c-37072f01559c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:18.477911 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-358ea347-d28e-478d-9e5c-37072f01559c None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:18.480168 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-358ea347-d28e-478d-9e5c-37072f01559c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:18.483030 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b47ee-9145-4181-837f-01752c6f0405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:18.484117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b47ee-9145-4181-837f-01752c6f0405 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:18.484563 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b47ee-9145-4181-837f-01752c6f0405 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:18.486079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:18.486260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b47ee-9145-4181-837f-01752c6f0405 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:18.486899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b47ee-9145-4181-837f-01752c6f0405 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:18.487408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b47ee-9145-4181-837f-01752c6f0405 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:18.489783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b47ee-9145-4181-837f-01752c6f0405 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:18.505740 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 522/1036] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Aug 8 16:33:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:18.512088 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b47ee-9145-4181-837f-01752c6f0405 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:18.596652 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 515/1037] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:33:18.622667 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ea347-d28e-478d-9e5c-37072f01559c None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:18.705798 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b47ee-9145-4181-837f-01752c6f0405 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:18.760089 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f976c-dac9-412c-a60e-064f349dc199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:18.760732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f976c-dac9-412c-a60e-064f349dc199 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:18.761048 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f976c-dac9-412c-a60e-064f349dc199 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:18.761317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f976c-dac9-412c-a60e-064f349dc199 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:18.762596 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ea347-d28e-478d-9e5c-37072f01559c None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:18.773972 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f976c-dac9-412c-a60e-064f349dc199 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:18.775471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f976c-dac9-412c-a60e-064f349dc199 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:18.792241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-901f976c-dac9-412c-a60e-064f349dc199 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:33:18.792241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:18.792241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:18.792241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:18.792241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:18.792241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:18.792241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:18.792241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:18.792241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:18.792241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:18.792241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:18.792241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:18.792241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:33:18.792241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:33:18.792241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:33:18.792241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:33:18.793578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:18.793578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:18.793578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:18.793578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:18.793578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:18.793578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:18.793578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:18.793578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:18.793578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:18.793578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:18.793578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:18.793578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:18.793578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:18.793578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:18.793578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:18.793578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:18.793578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:33:18.794682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:33:18.794682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:33:18.794682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:18.794682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 523/1038] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:33:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:33:18.799178 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f976c-dac9-412c-a60e-064f349dc199 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:18.817684 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:18.821112 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c318d4-a5a8-4795-9730-0ec8e5464cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:18.821768 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c318d4-a5a8-4795-9730-0ec8e5464cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:18.822051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c318d4-a5a8-4795-9730-0ec8e5464cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:18.822311 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c318d4-a5a8-4795-9730-0ec8e5464cd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:18.832020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:18.835967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c318d4-a5a8-4795-9730-0ec8e5464cd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:33:18.841316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-a4c318d4-a5a8-4795-9730-0ec8e5464cd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:33:18.841316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:18.841316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:18.841316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:18.841316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:18.841316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:18.841316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:18.841316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:18.841316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:18.841316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:18.841316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:18.841316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:18.841316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:33:18.841316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:33:18.841316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:33:18.841316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:33:18.843310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:33:18.843310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:33:18.843310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:33:18.843310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:33:18.843310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:18.843310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:18.843310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:18.843310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:18.843310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:18.843310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:18.843310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:18.843310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:18.843310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:18.843310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:18.843310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:18.843310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:18.844413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:18.844413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:18.844413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:18.844413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:18.844413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:33:18.844413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:33:18.844413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:33:18.844413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:33:18.844413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:33:18.844413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:33:18.844413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:33:18.844413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:18.844413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 516/1039] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:33:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:18.856114 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c318d4-a5a8-4795-9730-0ec8e5464cd2 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:18.975641 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f976c-dac9-412c-a60e-064f349dc199 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:19.042633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fada2b-ff59-4794-a6bc-c383b82829a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:19.043629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fada2b-ff59-4794-a6bc-c383b82829a7 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:19.043804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c318d4-a5a8-4795-9730-0ec8e5464cd2 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:19.044153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fada2b-ff59-4794-a6bc-c383b82829a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:19.044592 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fada2b-ff59-4794-a6bc-c383b82829a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:19.045697 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fada2b-ff59-4794-a6bc-c383b82829a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:19.046177 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fada2b-ff59-4794-a6bc-c383b82829a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:19.048357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fada2b-ff59-4794-a6bc-c383b82829a7 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:19.071425 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 524/1040] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:19.079985 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fada2b-ff59-4794-a6bc-c383b82829a7 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:19.109624 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d27fda-bba5-4359-b458-cf7a1e1999c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:19.110597 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d27fda-bba5-4359-b458-cf7a1e1999c8 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:19.111056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d27fda-bba5-4359-b458-cf7a1e1999c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:19.111496 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d27fda-bba5-4359-b458-cf7a1e1999c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:19.112602 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d27fda-bba5-4359-b458-cf7a1e1999c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:19.113045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d27fda-bba5-4359-b458-cf7a1e1999c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:19.115287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d27fda-bba5-4359-b458-cf7a1e1999c8 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:19.138609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 517/1041] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:33:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:19.143306 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d27fda-bba5-4359-b458-cf7a1e1999c8 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:19.209031 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:19.253130 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:19.265640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fada2b-ff59-4794-a6bc-c383b82829a7 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:19.325580 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d27fda-bba5-4359-b458-cf7a1e1999c8 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:19.329312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3c1c4-5d54-4811-9855-bef45c188468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:19.330257 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3c1c4-5d54-4811-9855-bef45c188468 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:19.330529 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3c1c4-5d54-4811-9855-bef45c188468 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:19.330861 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3c1c4-5d54-4811-9855-bef45c188468 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:19.343334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3c1c4-5d54-4811-9855-bef45c188468 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:19.345533 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3c1c4-5d54-4811-9855-bef45c188468 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:19.365975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-aae3c1c4-5d54-4811-9855-bef45c188468 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:33:19.365975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:19.365975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:19.365975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:19.365975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:19.365975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:19.365975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:19.365975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:19.365975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:19.365975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:19.365975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:19.365975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:19.365975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:33:19.365975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:33:19.365975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:33:19.365975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:33:19.368468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:19.368468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:19.368468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:33:19.368468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:33:19.368468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:33:19.368468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:33:19.368468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:19.368468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:19.368468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:19.368468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:19.368468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:19.368468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:19.368468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:19.368468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:19.368468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:19.368468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:19.368468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:19.368468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:19.370399 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:19.370399 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:19.370399 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:33:19.370399 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:33:19.370399 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:33:19.370399 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:33:19.370399 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:33:19.370399 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:33:19.370399 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:33:19.370399 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:19.370399 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 525/1042] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Aug 8 16:33:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:33:19.375588 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3c1c4-5d54-4811-9855-bef45c188468 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:19.398938 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39985656-7b8c-4f5e-ad69-ce710142e0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:19.399994 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39985656-7b8c-4f5e-ad69-ce710142e0d2 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:19.400310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39985656-7b8c-4f5e-ad69-ce710142e0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:19.400758 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39985656-7b8c-4f5e-ad69-ce710142e0d2 None admin] PATH_INFO: `/v3/users/3ffb95cc58574bc382a5b1fd7fb0742d` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:19.424082 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39985656-7b8c-4f5e-ad69-ce710142e0d2 None admin] RBAC: Authorizing `identity:get_user(user_id=3ffb95cc58574bc382a5b1fd7fb0742d)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:19.426368 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39985656-7b8c-4f5e-ad69-ce710142e0d2 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:19.429787 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 518/1043] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:33:19 2022] GET /identity/v3/users/3ffb95cc58574bc382a5b1fd7fb0742d => generated 285 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:19.440208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39985656-7b8c-4f5e-ad69-ce710142e0d2 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:19.534781 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3c1c4-5d54-4811-9855-bef45c188468 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:19.573783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:19.580676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062e2f3-82f4-4a05-ae54-bd26c00ea2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:19.581494 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062e2f3-82f4-4a05-ae54-bd26c00ea2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:19.581869 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062e2f3-82f4-4a05-ae54-bd26c00ea2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:19.582239 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062e2f3-82f4-4a05-ae54-bd26c00ea2b5 None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:19.583145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c062e2f3-82f4-4a05-ae54-bd26c00ea2b5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:19.586161 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c062e2f3-82f4-4a05-ae54-bd26c00ea2b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:19.586161 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c062e2f3-82f4-4a05-ae54-bd26c00ea2b5 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:19.608807 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 526/1044] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Aug 8 16:33:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:19.617968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062e2f3-82f4-4a05-ae54-bd26c00ea2b5 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:19.649709 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39985656-7b8c-4f5e-ad69-ce710142e0d2 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:19.685983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:19.707501 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d03f25-a77b-45c7-babf-c3a91d0c9893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:19.708121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d03f25-a77b-45c7-babf-c3a91d0c9893 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:19.708410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d03f25-a77b-45c7-babf-c3a91d0c9893 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:19.708662 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d03f25-a77b-45c7-babf-c3a91d0c9893 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:19.791370 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d03f25-a77b-45c7-babf-c3a91d0c9893 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:19.792451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062e2f3-82f4-4a05-ae54-bd26c00ea2b5 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:19.792720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d03f25-a77b-45c7-babf-c3a91d0c9893 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:19.852576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659a63b3-ad7f-4223-ac6c-cc29e94eefb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:19.853401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a63b3-ad7f-4223-ac6c-cc29e94eefb6 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:19.853823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a63b3-ad7f-4223-ac6c-cc29e94eefb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:19.854227 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a63b3-ad7f-4223-ac6c-cc29e94eefb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:19.939558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:19.958674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659a63b3-ad7f-4223-ac6c-cc29e94eefb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:19.962145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659a63b3-ad7f-4223-ac6c-cc29e94eefb6 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:19.967906 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 519/1045] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:33:19 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 16:33:19.973780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d03f25-a77b-45c7-babf-c3a91d0c9893 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:20.002895 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:20.124865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d03f25-a77b-45c7-babf-c3a91d0c9893 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:20.137689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 527/1046] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:33:19 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:33:20.146154 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659a63b3-ad7f-4223-ac6c-cc29e94eefb6 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:20.188443 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb427e9-2468-4d2c-be9d-4193c2c5dd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:20.189261 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb427e9-2468-4d2c-be9d-4193c2c5dd52 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:20.189663 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb427e9-2468-4d2c-be9d-4193c2c5dd52 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:20.190067 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb427e9-2468-4d2c-be9d-4193c2c5dd52 None admin] PATH_INFO: `/v3/projects/5f0fe56f23d54e67a6522dce3ed8c46a` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:20.191475 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb427e9-2468-4d2c-be9d-4193c2c5dd52 None admin] RBAC: Authorizing `identity:get_project(project_id=5f0fe56f23d54e67a6522dce3ed8c46a)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:20.193133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb427e9-2468-4d2c-be9d-4193c2c5dd52 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:20.201378 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 520/1047] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:33:19 2022] GET /identity/v3/projects/5f0fe56f23d54e67a6522dce3ed8c46a => generated 311 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:20.202588 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb427e9-2468-4d2c-be9d-4193c2c5dd52 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:20.312564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:20.328348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:20.331812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659a63b3-ad7f-4223-ac6c-cc29e94eefb6 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:20.347681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb427e9-2468-4d2c-be9d-4193c2c5dd52 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:20.397964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca1f2c-d106-4cdd-848c-97b381b1e25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:20.399380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca1f2c-d106-4cdd-848c-97b381b1e25a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:20.399824 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca1f2c-d106-4cdd-848c-97b381b1e25a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:20.400253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca1f2c-d106-4cdd-848c-97b381b1e25a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:20.411067 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c00dc4-a8fb-4a49-b21b-dff17eba509e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:20.411974 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c00dc4-a8fb-4a49-b21b-dff17eba509e None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:20.412381 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c00dc4-a8fb-4a49-b21b-dff17eba509e None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:20.412879 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c00dc4-a8fb-4a49-b21b-dff17eba509e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:20.413860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c00dc4-a8fb-4a49-b21b-dff17eba509e None admin] RBAC: Adding query filter params (role.id=e578ef831127463d9b48cc89fe20329c, scope.project.id=5f0fe56f23d54e67a6522dce3ed8c46a, user.id=3ffb95cc58574bc382a5b1fd7fb0742d) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:20.414342 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c00dc4-a8fb-4a49-b21b-dff17eba509e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:20.416232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c00dc4-a8fb-4a49-b21b-dff17eba509e None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:20.420497 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fca1f2c-d106-4cdd-848c-97b381b1e25a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:20.423545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fca1f2c-d106-4cdd-848c-97b381b1e25a None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:20.423755 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 521/1048] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:33:20 2022] GET /identity/v3/role_assignments?user.id=3ffb95cc58574bc382a5b1fd7fb0742d&scope.project.id=5f0fe56f23d54e67a6522dce3ed8c46a&role.id=e578ef831127463d9b48cc89fe20329c => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:20.453450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-5fca1f2c-d106-4cdd-848c-97b381b1e25a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:33:20.453450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:20.453450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:20.453450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:20.453450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:20.453450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:20.453450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:20.453450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:20.453450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:20.453450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:20.453450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:20.453450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:20.453450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:33:20.453450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:33:20.453450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:33:20.453450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:33:20.455445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:20.455445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:20.455445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:20.455445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:20.455445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:20.455445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:20.455445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:20.455445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:20.455445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:20.455445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:20.455445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:20.455445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:20.455445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:20.455445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:20.455445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:20.455445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:20.455445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:33:20.457237 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:33:20.457237 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:33:20.457237 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:20.457237 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 528/1049] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:33:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:33:20.462529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c00dc4-a8fb-4a49-b21b-dff17eba509e None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:20.512062 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea443ef0-769f-41d8-838f-9fabf10aa30d None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:20.512377 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea443ef0-769f-41d8-838f-9fabf10aa30d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:20.512639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea443ef0-769f-41d8-838f-9fabf10aa30d None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:20.514027 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 529/1050] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:33:20.519963 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095cb65-4852-4239-9fa2-19ef61f8e6b3 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:20.520220 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095cb65-4852-4239-9fa2-19ef61f8e6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:20.520501 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095cb65-4852-4239-9fa2-19ef61f8e6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:20.541562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-0095cb65-4852-4239-9fa2-19ef61f8e6b3 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:20.542873 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-0095cb65-4852-4239-9fa2-19ef61f8e6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:20.622705 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c00dc4-a8fb-4a49-b21b-dff17eba509e None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:20.653759 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 530/1051] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:33:20.674666 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cea7f7-b2e9-4deb-986e-b07022bc776d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:20.675455 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cea7f7-b2e9-4deb-986e-b07022bc776d None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:20.675625 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cea7f7-b2e9-4deb-986e-b07022bc776d None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:20.675814 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cea7f7-b2e9-4deb-986e-b07022bc776d None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:20.677091 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cea7f7-b2e9-4deb-986e-b07022bc776d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:20.677091 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cea7f7-b2e9-4deb-986e-b07022bc776d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:20.677622 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095cb65-4852-4239-9fa2-19ef61f8e6b3 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:20.678131 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cea7f7-b2e9-4deb-986e-b07022bc776d None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:20.694464 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 522/1052] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:20.700834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cea7f7-b2e9-4deb-986e-b07022bc776d None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:20.721145 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:20.782495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:20.833190 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095cb65-4852-4239-9fa2-19ef61f8e6b3 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:20.866298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cea7f7-b2e9-4deb-986e-b07022bc776d None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:20.892801 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c18110-a04b-45b3-8966-8904eb273b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:20.893828 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c18110-a04b-45b3-8966-8904eb273b12 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:20.894268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c18110-a04b-45b3-8966-8904eb273b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:20.894735 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c18110-a04b-45b3-8966-8904eb273b12 None admin] PATH_INFO: `/v3/services/image` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:20.895648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c18110-a04b-45b3-8966-8904eb273b12 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:20.898454 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c18110-a04b-45b3-8966-8904eb273b12 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:20.914621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-f9c18110-a04b-45b3-8966-8904eb273b12 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 16:33:20.914621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:20.914621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:33:20.914621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:33:20.914621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:33:20.914621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:33:20.914621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:33:20.914621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:33:20.914621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 16:33:20.914621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:20.914621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:33:20.914621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:20.914621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:20.914621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:20.914621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:20.914621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:20.914621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:20.915686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:20.915686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:20.915686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:20.915686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:20.915686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:20.915686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:20.915686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:33:20.915686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:33:20.915686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:33:20.915686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:33:20.915686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:20.915686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:20.915686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:20.915686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:20.915686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:20.915686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:20.916564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:20.916564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:20.916564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:20.916564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:20.916564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:20.916564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:20.916564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:20.916564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:20.916564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:20.916564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:20.916564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:33:20.916564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:33:20.916564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 16:33:20.916564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:20.917330 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 531/1053] 10.222.0.13 () {56 vars in 1105 bytes} [Mon Aug 8 16:33:20 2022] GET /identity/v3/services/image => generated 86 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:33:20.924586 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c18110-a04b-45b3-8966-8904eb273b12 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:20.934851 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49958a2-778a-49fd-be14-440d3f1b056e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:20.935506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49958a2-778a-49fd-be14-440d3f1b056e None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:20.935779 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49958a2-778a-49fd-be14-440d3f1b056e None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:20.936059 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49958a2-778a-49fd-be14-440d3f1b056e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:20.946474 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49958a2-778a-49fd-be14-440d3f1b056e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:20.949282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49958a2-778a-49fd-be14-440d3f1b056e None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:20.965793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-d49958a2-778a-49fd-be14-440d3f1b056e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:33:20.965793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:20.965793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:20.965793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:20.965793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:20.965793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:20.965793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:20.965793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:20.965793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:20.965793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:20.965793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:20.965793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:20.965793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:33:20.965793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:33:20.965793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:33:20.967892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:33:20.967892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:20.967892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:20.967892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:20.967892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:20.967892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:20.967892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:20.967892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:20.967892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:20.967892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:20.967892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:20.967892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:20.967892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:20.967892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:20.967892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:20.967892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:20.967892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:20.968877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:33:20.968877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:33:20.968877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:33:20.968877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:33:20.968877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:33:20.968877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:33:20.968877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:33:20.968877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:20.968877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 523/1054] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:33:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:33:20.973881 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49958a2-778a-49fd-be14-440d3f1b056e None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:21.068870 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:21.075102 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c18110-a04b-45b3-8966-8904eb273b12 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:21.115319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49958a2-778a-49fd-be14-440d3f1b056e None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:21.125144 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d383a-5132-45f3-aaa1-c5e78bfe8263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:21.125836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d383a-5132-45f3-aaa1-c5e78bfe8263 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:21.126130 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d383a-5132-45f3-aaa1-c5e78bfe8263 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:21.126418 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d383a-5132-45f3-aaa1-c5e78bfe8263 None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:21.127046 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73d383a-5132-45f3-aaa1-c5e78bfe8263 None admin] RBAC: Adding query filter params (name=image) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:21.127342 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73d383a-5132-45f3-aaa1-c5e78bfe8263 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:21.128916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73d383a-5132-45f3-aaa1-c5e78bfe8263 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:21.131399 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:21.143097 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 532/1055] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:33:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:21.144764 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d383a-5132-45f3-aaa1-c5e78bfe8263 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:21.164464 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c250554-0480-43ea-ab79-5501e16b9a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:21.165452 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c250554-0480-43ea-ab79-5501e16b9a66 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:21.165809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c250554-0480-43ea-ab79-5501e16b9a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:21.167335 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c250554-0480-43ea-ab79-5501e16b9a66 None admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:21.167422 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c250554-0480-43ea-ab79-5501e16b9a66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:21.167904 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c250554-0480-43ea-ab79-5501e16b9a66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:21.170755 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c250554-0480-43ea-ab79-5501e16b9a66 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:21.188969 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 524/1056] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:33:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:21.195796 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c250554-0480-43ea-ab79-5501e16b9a66 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:21.294149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d383a-5132-45f3-aaa1-c5e78bfe8263 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:21.346762 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ed3b8-f3d8-4779-834d-1d8846e3e8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:21.347875 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ed3b8-f3d8-4779-834d-1d8846e3e8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:21.348354 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ed3b8-f3d8-4779-834d-1d8846e3e8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:21.348815 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ed3b8-f3d8-4779-834d-1d8846e3e8d4 None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:21.349986 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ed3b8-f3d8-4779-834d-1d8846e3e8d4 None admin] RBAC: Adding query filter params (type=image) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:21.350698 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ed3b8-f3d8-4779-834d-1d8846e3e8d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:21.354194 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ed3b8-f3d8-4779-834d-1d8846e3e8d4 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:21.372525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 533/1057] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:33:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:21.374285 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ed3b8-f3d8-4779-834d-1d8846e3e8d4 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:21.382928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c250554-0480-43ea-ab79-5501e16b9a66 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:21.428299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:21.434425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517869bb-481b-4a92-97c5-f4b8f7ab4564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:21.435085 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517869bb-481b-4a92-97c5-f4b8f7ab4564 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:21.435571 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517869bb-481b-4a92-97c5-f4b8f7ab4564 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:21.435726 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517869bb-481b-4a92-97c5-f4b8f7ab4564 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:21.436449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517869bb-481b-4a92-97c5-f4b8f7ab4564 None admin] RBAC: Adding query filter params (role.id=16940dcad5d247d0b43da26eb2b69cbc, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=9daa7489bced467984d39e8b8ac49d13) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:21.436752 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517869bb-481b-4a92-97c5-f4b8f7ab4564 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:21.438308 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517869bb-481b-4a92-97c5-f4b8f7ab4564 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:21.446211 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 525/1058] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:33:21 2022] GET /identity/v3/role_assignments?user.id=9daa7489bced467984d39e8b8ac49d13&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=16940dcad5d247d0b43da26eb2b69cbc => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:21.514309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:21.595161 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ed3b8-f3d8-4779-834d-1d8846e3e8d4 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:21.660103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda78744-0885-4794-a99e-10d3ce02c96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:21.661302 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda78744-0885-4794-a99e-10d3ce02c96c None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:21.661751 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda78744-0885-4794-a99e-10d3ce02c96c None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:21.662201 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda78744-0885-4794-a99e-10d3ce02c96c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:21.663455 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda78744-0885-4794-a99e-10d3ce02c96c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:21.666781 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda78744-0885-4794-a99e-10d3ce02c96c None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:21.761477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 534/1059] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:33:21 2022] POST /identity/v3/endpoints => generated 327 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:33:23.552830 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43909a98-cef3-4e59-b0ca-7322ee16dcd2 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:23.553600 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43909a98-cef3-4e59-b0ca-7322ee16dcd2 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:23.553600 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43909a98-cef3-4e59-b0ca-7322ee16dcd2 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:23.555638 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 526/1060] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:33:23.561170 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ff451-58e4-4e97-a417-6512caada61d None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:23.561944 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ff451-58e4-4e97-a417-6512caada61d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:23.561944 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ff451-58e4-4e97-a417-6512caada61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:23.581692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-847ff451-58e4-4e97-a417-6512caada61d None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:23.583073 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-847ff451-58e4-4e97-a417-6512caada61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:23.722530 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 535/1061] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:33:23.736843 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e3634-84fb-494c-b79d-31926f10b9ba None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:23.737320 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e3634-84fb-494c-b79d-31926f10b9ba None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:23.737791 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e3634-84fb-494c-b79d-31926f10b9ba None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:23.740413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 527/1062] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:33:23.749762 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c8917-06d6-4a97-aefb-1f7df85f4200 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:23.750200 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c8917-06d6-4a97-aefb-1f7df85f4200 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:23.750704 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c8917-06d6-4a97-aefb-1f7df85f4200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:23.762014 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e3634-84fb-494c-b79d-31926f10b9ba None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:23.781550 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-a17c8917-06d6-4a97-aefb-1f7df85f4200 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:23.783398 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-a17c8917-06d6-4a97-aefb-1f7df85f4200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:23.854302 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:23.885335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 536/1063] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:33:23.909490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c8917-06d6-4a97-aefb-1f7df85f4200 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:23.935894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e3634-84fb-494c-b79d-31926f10b9ba None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:23.982212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fdf3d-74a6-4d80-be15-80df704111d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:23.983419 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fdf3d-74a6-4d80-be15-80df704111d8 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:23.983971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fdf3d-74a6-4d80-be15-80df704111d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:23.984545 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fdf3d-74a6-4d80-be15-80df704111d8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:24.004776 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643fdf3d-74a6-4d80-be15-80df704111d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:33:24.014677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-643fdf3d-74a6-4d80-be15-80df704111d8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:33:24.014677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:24.014677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:24.014677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:24.014677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:24.014677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:24.014677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:24.014677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:24.014677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:24.014677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:24.014677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:24.014677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:24.014677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:33:24.014677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:33:24.014677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:33:24.014677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:33:24.024369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:33:24.024369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:33:24.024369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:33:24.024369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:33:24.024369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:24.024369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:24.024369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:24.024369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:24.024369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:24.024369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:24.024369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:24.024369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:24.024369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:24.024369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:24.024369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:24.024369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:24.026045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:24.026045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:24.026045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:24.026045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:24.026045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:33:24.026045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:33:24.026045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:33:24.026045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:33:24.026045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:33:24.026045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:33:24.026045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:33:24.026045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:24.026045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 528/1064] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Aug 8 16:33:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:24.028713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fdf3d-74a6-4d80-be15-80df704111d8 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:24.072333 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c8917-06d6-4a97-aefb-1f7df85f4200 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:24.126398 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:24.128136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918f842-600f-435a-aad2-0d16ba722a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:24.129136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918f842-600f-435a-aad2-0d16ba722a4b None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:24.129585 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918f842-600f-435a-aad2-0d16ba722a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:24.130044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918f842-600f-435a-aad2-0d16ba722a4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:24.147350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9918f842-600f-435a-aad2-0d16ba722a4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:33:24.153146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-9918f842-600f-435a-aad2-0d16ba722a4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:33:24.153146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:24.153146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:24.153146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:24.153146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:24.153146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:24.153146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:24.153146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:24.153146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:24.153146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:24.153146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:24.153146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:24.153146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:33:24.153146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:33:24.153146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:33:24.153146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:33:24.154836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:33:24.154836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:33:24.154836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:33:24.154836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:33:24.154836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:24.154836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:24.154836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:24.154836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:24.154836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:24.154836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:24.154836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:24.154836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:24.154836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:24.154836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:24.154836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:24.154836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:24.156626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:24.156626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:24.156626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:24.156626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:24.156626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:33:24.156626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:33:24.156626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:33:24.156626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:33:24.156626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:33:24.156626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:33:24.156626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:33:24.156626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:24.156626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 537/1065] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:33:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:33:24.163419 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918f842-600f-435a-aad2-0d16ba722a4b None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:24.202851 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fdf3d-74a6-4d80-be15-80df704111d8 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:24.208561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:24.246378 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ed050-984f-4f2a-bd60-462d55794924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:24.247437 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ed050-984f-4f2a-bd60-462d55794924 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:24.247437 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ed050-984f-4f2a-bd60-462d55794924 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:24.247620 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ed050-984f-4f2a-bd60-462d55794924 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:24.248221 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779ed050-984f-4f2a-bd60-462d55794924 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:24.248519 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779ed050-984f-4f2a-bd60-462d55794924 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:24.249976 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779ed050-984f-4f2a-bd60-462d55794924 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:24.265701 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 529/1066] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Aug 8 16:33:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:24.266805 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ed050-984f-4f2a-bd60-462d55794924 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:24.296676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918f842-600f-435a-aad2-0d16ba722a4b None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:24.354086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3fee2-5650-4e89-9c40-3fdd154bee0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:24.354734 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3fee2-5650-4e89-9c40-3fdd154bee0c None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:24.355013 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3fee2-5650-4e89-9c40-3fdd154bee0c None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:24.355282 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3fee2-5650-4e89-9c40-3fdd154bee0c None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:24.355959 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff3fee2-5650-4e89-9c40-3fdd154bee0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:24.356254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff3fee2-5650-4e89-9c40-3fdd154bee0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:24.357636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff3fee2-5650-4e89-9c40-3fdd154bee0c None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:24.367379 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 538/1067] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:33:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:24.373169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3fee2-5650-4e89-9c40-3fdd154bee0c None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:24.390588 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ed050-984f-4f2a-bd60-462d55794924 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:24.413105 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:24.439657 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404360f-b8d4-49f0-9aa0-a3ba3a8b25de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:24.440560 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404360f-b8d4-49f0-9aa0-a3ba3a8b25de None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:24.441029 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404360f-b8d4-49f0-9aa0-a3ba3a8b25de None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:24.441434 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404360f-b8d4-49f0-9aa0-a3ba3a8b25de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:24.456752 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d404360f-b8d4-49f0-9aa0-a3ba3a8b25de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:24.458213 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d404360f-b8d4-49f0-9aa0-a3ba3a8b25de None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:24.480338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-d404360f-b8d4-49f0-9aa0-a3ba3a8b25de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:33:24.480338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:24.480338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:24.480338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:24.480338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:24.480338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:24.480338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:24.480338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:24.480338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:24.480338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:24.480338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:24.480338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:24.480338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:33:24.480338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:33:24.480338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:33:24.480338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:33:24.481722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:24.481722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:24.481722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:24.481722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:24.481722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:24.481722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:24.481722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:24.481722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:24.481722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:24.481722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:24.481722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:24.481722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:24.481722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:24.481722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:24.481722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:24.481722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:24.481722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:33:24.482724 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:33:24.482724 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:33:24.482724 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:24.482724 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 530/1068] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:33:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:24.487480 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404360f-b8d4-49f0-9aa0-a3ba3a8b25de None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:24.504905 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:24.509668 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3fee2-5650-4e89-9c40-3fdd154bee0c None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:24.548839 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9918e2-9d9b-4ed8-b4b0-62a5e7e8658a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:24.549477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9918e2-9d9b-4ed8-b4b0-62a5e7e8658a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:24.549756 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9918e2-9d9b-4ed8-b4b0-62a5e7e8658a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:24.550028 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9918e2-9d9b-4ed8-b4b0-62a5e7e8658a None admin] PATH_INFO: `/v3/users/3ffb95cc58574bc382a5b1fd7fb0742d` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:24.551237 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9918e2-9d9b-4ed8-b4b0-62a5e7e8658a None admin] RBAC: Authorizing `identity:get_user(user_id=3ffb95cc58574bc382a5b1fd7fb0742d)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:24.552769 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9918e2-9d9b-4ed8-b4b0-62a5e7e8658a None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:24.558627 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 539/1069] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:33:24 2022] GET /identity/v3/users/3ffb95cc58574bc382a5b1fd7fb0742d => generated 285 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:24.560166 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9918e2-9d9b-4ed8-b4b0-62a5e7e8658a None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:24.607645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404360f-b8d4-49f0-9aa0-a3ba3a8b25de None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:24.674531 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9918e2-9d9b-4ed8-b4b0-62a5e7e8658a None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:24.711845 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ceb8a8-da54-4552-8f7e-c6d79dfa56cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:24.716747 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ceb8a8-da54-4552-8f7e-c6d79dfa56cc None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:24.716747 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ceb8a8-da54-4552-8f7e-c6d79dfa56cc None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:24.716747 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ceb8a8-da54-4552-8f7e-c6d79dfa56cc None admin] PATH_INFO: `/v3/projects/5f0fe56f23d54e67a6522dce3ed8c46a` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:24.716747 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ceb8a8-da54-4552-8f7e-c6d79dfa56cc None admin] RBAC: Authorizing `identity:get_project(project_id=5f0fe56f23d54e67a6522dce3ed8c46a)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:24.716747 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ceb8a8-da54-4552-8f7e-c6d79dfa56cc None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:24.717984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 540/1070] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:33:24 2022] GET /identity/v3/projects/5f0fe56f23d54e67a6522dce3ed8c46a => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:24.719083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef5362-188b-4dea-a51b-a4153f16e6f2 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:24.719346 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef5362-188b-4dea-a51b-a4153f16e6f2 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:24.719611 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef5362-188b-4dea-a51b-a4153f16e6f2 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:24.720855 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 541/1071] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:33:24.725817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef5362-188b-4dea-a51b-a4153f16e6f2 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:24.750720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d4396-dc77-4692-8862-070a76c09bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:24.751328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d4396-dc77-4692-8862-070a76c09bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:24.751602 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d4396-dc77-4692-8862-070a76c09bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:24.753377 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d4396-dc77-4692-8862-070a76c09bc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:24.753377 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d4396-dc77-4692-8862-070a76c09bc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:24.753377 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d4396-dc77-4692-8862-070a76c09bc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:24.754162 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d4396-dc77-4692-8862-070a76c09bc5 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:24.766999 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 531/1072] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:24.768277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1f5f1-c2fc-459a-8470-b283eded5fd1 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:24.768556 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1f5f1-c2fc-459a-8470-b283eded5fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:24.768898 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1f5f1-c2fc-459a-8470-b283eded5fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:24.782520 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:24.837805 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-bbb1f5f1-c2fc-459a-8470-b283eded5fd1 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:24.839152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-bbb1f5f1-c2fc-459a-8470-b283eded5fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:24.869325 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:24.901545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef5362-188b-4dea-a51b-a4153f16e6f2 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:24.917237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 532/1073] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:33:24.918095 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb1f5f1-c2fc-459a-8470-b283eded5fd1 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:24.944425 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20438b4e-98d5-4b76-9f5f-4f8398d9016b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:24.945551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20438b4e-98d5-4b76-9f5f-4f8398d9016b None admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:24.946011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20438b4e-98d5-4b76-9f5f-4f8398d9016b None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:24.946450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20438b4e-98d5-4b76-9f5f-4f8398d9016b None admin] PATH_INFO: `/v3/projects/5f0fe56f23d54e67a6522dce3ed8c46a/users/3ffb95cc58574bc382a5b1fd7fb0742d/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:24.949591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20438b4e-98d5-4b76-9f5f-4f8398d9016b None admin] RBAC: Authorizing `identity:create_grant(project_id=5f0fe56f23d54e67a6522dce3ed8c46a, user_id=3ffb95cc58574bc382a5b1fd7fb0742d, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:24.951657 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20438b4e-98d5-4b76-9f5f-4f8398d9016b None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:25.025927 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 542/1074] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:33:24 2022] PUT /identity/v3/projects/5f0fe56f23d54e67a6522dce3ed8c46a/users/3ffb95cc58574bc382a5b1fd7fb0742d/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:33:25.043552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20438b4e-98d5-4b76-9f5f-4f8398d9016b None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:25.061650 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb1f5f1-c2fc-459a-8470-b283eded5fd1 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:25.111183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb79b730-44b7-4d50-aa95-cdbac25f01fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:25.111881 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79b730-44b7-4d50-aa95-cdbac25f01fc None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:25.112158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79b730-44b7-4d50-aa95-cdbac25f01fc None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:25.112428 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79b730-44b7-4d50-aa95-cdbac25f01fc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:25.122773 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79b730-44b7-4d50-aa95-cdbac25f01fc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:25.125520 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79b730-44b7-4d50-aa95-cdbac25f01fc None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:25.140764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-cb79b730-44b7-4d50-aa95-cdbac25f01fc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:33:25.140764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:25.140764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:25.140764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:25.140764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:25.140764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:25.140764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:25.140764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:25.140764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:25.140764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:25.140764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:25.140764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:25.140764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:33:25.140764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:33:25.140764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:33:25.140764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:33:25.141916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:25.141916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:25.141916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:33:25.141916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:33:25.141916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:33:25.141916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:33:25.141916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:25.141916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:25.141916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:25.141916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:25.141916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:25.141916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:25.141916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:25.141916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:25.141916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:25.141916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:25.141916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:25.141916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:25.142952 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:25.142952 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:25.142952 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:33:25.142952 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:33:25.142952 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:33:25.142952 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:33:25.142952 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:33:25.142952 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:33:25.142952 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:33:25.142952 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:25.142952 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 533/1075] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Aug 8 16:33:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:25.148795 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb79b730-44b7-4d50-aa95-cdbac25f01fc None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:25.174637 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:25.187839 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:25.198742 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20438b4e-98d5-4b76-9f5f-4f8398d9016b None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:25.250636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c993254-dab3-4a5d-97bf-f9f0f5e89e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:25.251635 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c993254-dab3-4a5d-97bf-f9f0f5e89e73 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:25.252091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c993254-dab3-4a5d-97bf-f9f0f5e89e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:25.252535 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c993254-dab3-4a5d-97bf-f9f0f5e89e73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:25.285376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb79b730-44b7-4d50-aa95-cdbac25f01fc None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:25.326073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1010e88-d3e5-4751-8e49-ff4a1fc181b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:25.326724 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1010e88-d3e5-4751-8e49-ff4a1fc181b8 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:25.327001 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1010e88-d3e5-4751-8e49-ff4a1fc181b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:25.327269 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1010e88-d3e5-4751-8e49-ff4a1fc181b8 None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:25.327995 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1010e88-d3e5-4751-8e49-ff4a1fc181b8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:25.328299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1010e88-d3e5-4751-8e49-ff4a1fc181b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:25.329915 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1010e88-d3e5-4751-8e49-ff4a1fc181b8 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:25.348598 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c993254-dab3-4a5d-97bf-f9f0f5e89e73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:25.350547 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c993254-dab3-4a5d-97bf-f9f0f5e89e73 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:25.357834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 534/1076] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Aug 8 16:33:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:25.359275 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1010e88-d3e5-4751-8e49-ff4a1fc181b8 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:25.466235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:25.474408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:25.484130 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 543/1077] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:33:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:33:25.489889 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c993254-dab3-4a5d-97bf-f9f0f5e89e73 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:25.490211 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1010e88-d3e5-4751-8e49-ff4a1fc181b8 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:25.538585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff42b8-0844-4642-b1ea-9a365064ae4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:25.539657 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff42b8-0844-4642-b1ea-9a365064ae4d None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:25.539657 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff42b8-0844-4642-b1ea-9a365064ae4d None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:25.539841 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff42b8-0844-4642-b1ea-9a365064ae4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:25.552187 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff42b8-0844-4642-b1ea-9a365064ae4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:25.553644 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff42b8-0844-4642-b1ea-9a365064ae4d None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:25.576634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-24ff42b8-0844-4642-b1ea-9a365064ae4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:33:25.576634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:25.576634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:25.576634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:25.576634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:25.576634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:25.576634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:25.576634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:25.576634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:25.576634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:25.576634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:25.576634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:25.576634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:33:25.576634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:33:25.576634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:33:25.576634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:33:25.577723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:25.577723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:25.577723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:25.577723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:25.577723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:25.577723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:25.577723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:25.577723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:25.577723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:25.577723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:25.577723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:25.577723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:25.577723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:25.577723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:25.577723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:25.577723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:25.577723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:33:25.583456 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:33:25.583456 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:33:25.583456 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:25.583456 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 535/1078] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:33:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:25.583703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff42b8-0844-4642-b1ea-9a365064ae4d None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:25.613573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c993254-dab3-4a5d-97bf-f9f0f5e89e73 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:25.659885 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40432635-ae7f-4de4-a6af-059617f19c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:25.660387 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40432635-ae7f-4de4-a6af-059617f19c14 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:25.660667 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40432635-ae7f-4de4-a6af-059617f19c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:25.660956 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40432635-ae7f-4de4-a6af-059617f19c14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:25.673622 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40432635-ae7f-4de4-a6af-059617f19c14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:25.675247 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40432635-ae7f-4de4-a6af-059617f19c14 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:25.693283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-40432635-ae7f-4de4-a6af-059617f19c14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:33:25.693283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:25.693283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:25.693283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:25.693283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:25.693283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:25.693283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:25.693283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:25.693283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:25.693283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:25.693283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:25.693283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:25.693283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:33:25.693283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:33:25.693283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:33:25.693283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:33:25.694542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:25.694542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:25.694542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:25.694542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:25.694542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:25.694542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:25.694542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:25.694542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:25.694542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:25.694542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:25.694542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:25.694542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:25.694542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:25.694542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:25.694542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:25.694542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:25.694542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:33:25.696222 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:33:25.696222 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:33:25.696222 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:25.698354 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 544/1079] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:33:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:33:25.701257 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40432635-ae7f-4de4-a6af-059617f19c14 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:25.721361 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff42b8-0844-4642-b1ea-9a365064ae4d None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:25.755614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:25.758829 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:25.762984 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad21392-00f5-4497-ba2d-d101567e9f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:25.763601 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad21392-00f5-4497-ba2d-d101567e9f5a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:25.763876 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad21392-00f5-4497-ba2d-d101567e9f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:25.764142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad21392-00f5-4497-ba2d-d101567e9f5a None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:25.764825 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad21392-00f5-4497-ba2d-d101567e9f5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:25.765119 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad21392-00f5-4497-ba2d-d101567e9f5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:25.766524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad21392-00f5-4497-ba2d-d101567e9f5a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:25.784397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 536/1080] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:25.793116 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad21392-00f5-4497-ba2d-d101567e9f5a None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:25.826160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40432635-ae7f-4de4-a6af-059617f19c14 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:25.871451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e186f1-19c8-4c02-a5b5-cac993258651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:25.872076 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e186f1-19c8-4c02-a5b5-cac993258651 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:25.872354 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e186f1-19c8-4c02-a5b5-cac993258651 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:25.872622 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e186f1-19c8-4c02-a5b5-cac993258651 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:25.873268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e186f1-19c8-4c02-a5b5-cac993258651 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:25.873574 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e186f1-19c8-4c02-a5b5-cac993258651 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:25.874978 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e186f1-19c8-4c02-a5b5-cac993258651 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:25.890087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 545/1081] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:25.937433 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad21392-00f5-4497-ba2d-d101567e9f5a None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:25.989326 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790e4c4-95af-4fda-aee9-346d49554678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:25.990157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790e4c4-95af-4fda-aee9-346d49554678 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:25.990709 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790e4c4-95af-4fda-aee9-346d49554678 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:25.991159 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790e4c4-95af-4fda-aee9-346d49554678 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:26.003552 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790e4c4-95af-4fda-aee9-346d49554678 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:26.006118 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790e4c4-95af-4fda-aee9-346d49554678 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:26.026771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-f790e4c4-95af-4fda-aee9-346d49554678 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:33:26.026771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:26.026771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:26.026771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:26.026771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:26.026771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:26.026771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:26.026771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:26.026771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:26.026771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:26.026771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:26.026771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:26.026771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:33:26.026771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:33:26.026771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:33:26.027806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:33:26.027806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:26.027806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:26.027806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:26.027806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:26.027806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:26.027806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:26.027806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:26.027806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:26.027806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:26.027806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:26.027806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:26.027806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:26.027806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:26.027806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:26.027806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:26.027806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:26.028736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:33:26.028736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:33:26.028736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:33:26.028736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:33:26.028736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:33:26.028736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:33:26.028736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:33:26.028736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:26.035936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 537/1082] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:33:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:33:26.036328 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e186f1-19c8-4c02-a5b5-cac993258651 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:26.190296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e186f1-19c8-4c02-a5b5-cac993258651 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:26.201381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:26.239301 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21bdc1-2aa8-41a0-94b7-0036239b2352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:26.240348 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21bdc1-2aa8-41a0-94b7-0036239b2352 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:26.240755 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21bdc1-2aa8-41a0-94b7-0036239b2352 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:26.241201 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21bdc1-2aa8-41a0-94b7-0036239b2352 None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:26.242617 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21bdc1-2aa8-41a0-94b7-0036239b2352 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:26.243126 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21bdc1-2aa8-41a0-94b7-0036239b2352 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:26.244991 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21bdc1-2aa8-41a0-94b7-0036239b2352 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:26.259197 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 546/1083] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:33:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:26.265580 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790e4c4-95af-4fda-aee9-346d49554678 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:26.321618 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:26.376133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790e4c4-95af-4fda-aee9-346d49554678 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:26.411449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf740800-b17c-4729-80ec-720ee160c618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:26.412120 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf740800-b17c-4729-80ec-720ee160c618 None admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:26.412415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf740800-b17c-4729-80ec-720ee160c618 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:26.412677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf740800-b17c-4729-80ec-720ee160c618 None admin] PATH_INFO: `/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/9daa7489bced467984d39e8b8ac49d13/roles/16940dcad5d247d0b43da26eb2b69cbc` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:26.416039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf740800-b17c-4729-80ec-720ee160c618 None admin] RBAC: Authorizing `identity:create_grant(project_id=56ab2fc79d964f4d86edefb4d1b2c993, user_id=9daa7489bced467984d39e8b8ac49d13, role_id=16940dcad5d247d0b43da26eb2b69cbc)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:26.417522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf740800-b17c-4729-80ec-720ee160c618 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:26.489921 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 538/1084] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:33:26 2022] PUT /identity/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/9daa7489bced467984d39e8b8ac49d13/roles/16940dcad5d247d0b43da26eb2b69cbc => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:33:27.127188 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbe922-d679-4349-bd56-37fd2d5b6d46 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:27.128036 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbe922-d679-4349-bd56-37fd2d5b6d46 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:27.128212 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbe922-d679-4349-bd56-37fd2d5b6d46 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:27.129583 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 547/1085] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:33:27.135907 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce41def-77ec-4dc7-b908-6b89f2577fc7 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:27.136425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce41def-77ec-4dc7-b908-6b89f2577fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:27.136425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce41def-77ec-4dc7-b908-6b89f2577fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:27.155595 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-dce41def-77ec-4dc7-b908-6b89f2577fc7 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:27.156801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-dce41def-77ec-4dc7-b908-6b89f2577fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:27.227095 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 539/1086] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:33:27.251056 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bbe922-d679-4349-bd56-37fd2d5b6d46 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:27.356492 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bbe922-d679-4349-bd56-37fd2d5b6d46 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:27.391937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110af9c-22bc-49e6-95dc-0202cc43561a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:27.392577 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110af9c-22bc-49e6-95dc-0202cc43561a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:27.392856 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110af9c-22bc-49e6-95dc-0202cc43561a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:27.393146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110af9c-22bc-49e6-95dc-0202cc43561a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:27.404464 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110af9c-22bc-49e6-95dc-0202cc43561a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:33:27.409169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-4110af9c-22bc-49e6-95dc-0202cc43561a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:33:27.409169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:27.409169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:27.409169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:27.409169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:27.409169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:27.409169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:27.409169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:27.409169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:27.409169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:27.409169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:27.409169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:27.409169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:33:27.409169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:33:27.409169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:33:27.409169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:33:27.411111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:33:27.411111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:33:27.411111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:33:27.411111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:33:27.411111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:27.411111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:27.411111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:27.411111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:27.411111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:27.411111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:27.411111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:27.411111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:27.411111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:27.411111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:27.411111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:27.411111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:27.412895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:27.412895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:27.412895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:27.412895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:27.412895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:33:27.412895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:33:27.412895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:33:27.412895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:33:27.412895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:33:27.412895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:33:27.412895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:33:27.412895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:27.416247 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 548/1087] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:33:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:27.417688 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce41def-77ec-4dc7-b908-6b89f2577fc7 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:27.463127 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:27.522670 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce41def-77ec-4dc7-b908-6b89f2577fc7 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:27.560365 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7015cc0-47a2-41a0-a010-a02b35160490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:27.560978 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7015cc0-47a2-41a0-a010-a02b35160490 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:27.561250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7015cc0-47a2-41a0-a010-a02b35160490 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:27.561515 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7015cc0-47a2-41a0-a010-a02b35160490 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:27.562178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7015cc0-47a2-41a0-a010-a02b35160490 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:27.562476 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7015cc0-47a2-41a0-a010-a02b35160490 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:27.563889 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7015cc0-47a2-41a0-a010-a02b35160490 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:27.577553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 540/1088] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:33:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:27.578812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110af9c-22bc-49e6-95dc-0202cc43561a None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:27.609826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:27.690795 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110af9c-22bc-49e6-95dc-0202cc43561a None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:27.733618 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb8c9d-50d2-437f-87b6-170886028d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:27.734342 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb8c9d-50d2-437f-87b6-170886028d2b None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:27.734710 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb8c9d-50d2-437f-87b6-170886028d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:27.735051 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb8c9d-50d2-437f-87b6-170886028d2b None admin] PATH_INFO: `/v3/users/3ffb95cc58574bc382a5b1fd7fb0742d` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:27.736279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb8c9d-50d2-437f-87b6-170886028d2b None admin] RBAC: Authorizing `identity:get_user(user_id=3ffb95cc58574bc382a5b1fd7fb0742d)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:27.737807 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb8c9d-50d2-437f-87b6-170886028d2b None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:27.747299 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 549/1089] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:33:27 2022] GET /identity/v3/users/3ffb95cc58574bc382a5b1fd7fb0742d => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:27.748509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7015cc0-47a2-41a0-a010-a02b35160490 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:27.863009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7015cc0-47a2-41a0-a010-a02b35160490 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:27.865135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:27.904501 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807ddb7-5146-4963-bb03-cca3e4bfd79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:27.905224 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807ddb7-5146-4963-bb03-cca3e4bfd79c None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:27.905608 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807ddb7-5146-4963-bb03-cca3e4bfd79c None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:27.905972 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807ddb7-5146-4963-bb03-cca3e4bfd79c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:27.985291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0807ddb7-5146-4963-bb03-cca3e4bfd79c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:27.986807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0807ddb7-5146-4963-bb03-cca3e4bfd79c None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:28.102095 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 541/1090] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:33:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 16:33:28.108389 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb8c9d-50d2-437f-87b6-170886028d2b None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:28.214221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:28.222278 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb8c9d-50d2-437f-87b6-170886028d2b None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:28.264289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba7eab-74d5-46e6-9586-46da9de74c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:28.265066 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba7eab-74d5-46e6-9586-46da9de74c49 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:28.265432 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba7eab-74d5-46e6-9586-46da9de74c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:28.265812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba7eab-74d5-46e6-9586-46da9de74c49 None admin] PATH_INFO: `/v3/projects/5f0fe56f23d54e67a6522dce3ed8c46a` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:28.267220 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba7eab-74d5-46e6-9586-46da9de74c49 None admin] RBAC: Authorizing `identity:get_project(project_id=5f0fe56f23d54e67a6522dce3ed8c46a)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:28.268770 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba7eab-74d5-46e6-9586-46da9de74c49 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:28.270544 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 550/1091] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:33:28 2022] GET /identity/v3/projects/5f0fe56f23d54e67a6522dce3ed8c46a => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:28.277593 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807ddb7-5146-4963-bb03-cca3e4bfd79c None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:28.283276 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:28.394557 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807ddb7-5146-4963-bb03-cca3e4bfd79c None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:28.434290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b2dc3-36a8-4607-90e0-f581a14b8bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:28.434889 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b2dc3-36a8-4607-90e0-f581a14b8bbb None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:28.435164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b2dc3-36a8-4607-90e0-f581a14b8bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:28.435431 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b2dc3-36a8-4607-90e0-f581a14b8bbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:28.436164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b2dc3-36a8-4607-90e0-f581a14b8bbb None admin] RBAC: Adding query filter params (role.id=e578ef831127463d9b48cc89fe20329c, scope.project.id=5f0fe56f23d54e67a6522dce3ed8c46a, user.id=3ffb95cc58574bc382a5b1fd7fb0742d) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:28.436470 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b2dc3-36a8-4607-90e0-f581a14b8bbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:28.437925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b2dc3-36a8-4607-90e0-f581a14b8bbb None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:28.444931 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 542/1092] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:33:28 2022] GET /identity/v3/role_assignments?user.id=3ffb95cc58574bc382a5b1fd7fb0742d&scope.project.id=5f0fe56f23d54e67a6522dce3ed8c46a&role.id=e578ef831127463d9b48cc89fe20329c => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:28.582623 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb14f97-df39-46cf-80c0-f2b8d2fbdd55 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:28.583023 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb14f97-df39-46cf-80c0-f2b8d2fbdd55 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:28.583531 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb14f97-df39-46cf-80c0-f2b8d2fbdd55 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:28.586807 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 551/1093] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:33:28.593342 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44b7f2-f16f-4c2a-8332-8c4168a024e1 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:28.593674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44b7f2-f16f-4c2a-8332-8c4168a024e1 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:28.593970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44b7f2-f16f-4c2a-8332-8c4168a024e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:28.615437 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-6f44b7f2-f16f-4c2a-8332-8c4168a024e1 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:28.616890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-6f44b7f2-f16f-4c2a-8332-8c4168a024e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:28.700528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:28.708911 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 543/1094] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:33:28.733846 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb14f97-df39-46cf-80c0-f2b8d2fbdd55 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:28.850887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb14f97-df39-46cf-80c0-f2b8d2fbdd55 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:28.889408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee592ea-d89f-4c8d-8d9b-429d5a6c9b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:28.890074 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee592ea-d89f-4c8d-8d9b-429d5a6c9b32 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:28.890351 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee592ea-d89f-4c8d-8d9b-429d5a6c9b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:28.890646 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee592ea-d89f-4c8d-8d9b-429d5a6c9b32 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:28.904057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee592ea-d89f-4c8d-8d9b-429d5a6c9b32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:33:28.908776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-3ee592ea-d89f-4c8d-8d9b-429d5a6c9b32 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:33:28.908776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:28.908776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:28.908776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:28.908776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:28.908776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:28.908776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:28.908776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:28.908776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:28.908776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:28.908776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:28.908776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:28.908776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:33:28.908776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:33:28.908776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:33:28.908776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:33:28.911014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:33:28.911014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:33:28.911014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:33:28.911014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:33:28.911014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:28.911014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:28.911014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:28.911014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:28.911014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:28.911014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:28.911014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:28.911014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:28.911014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:28.911014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:28.911014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:28.911014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:28.912965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:28.912965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:28.912965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:28.912965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:28.912965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:33:28.912965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:33:28.912965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:33:28.912965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:33:28.912965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:33:28.912965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:33:28.912965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:33:28.912965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:28.914762 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 552/1095] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Aug 8 16:33:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:33:28.917268 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44b7f2-f16f-4c2a-8332-8c4168a024e1 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:29.050462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44b7f2-f16f-4c2a-8332-8c4168a024e1 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:29.088622 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835841cf-81ce-40fd-b00e-b457e2175cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:29.089233 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835841cf-81ce-40fd-b00e-b457e2175cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:29.089502 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835841cf-81ce-40fd-b00e-b457e2175cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:29.089766 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835841cf-81ce-40fd-b00e-b457e2175cb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:29.090437 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835841cf-81ce-40fd-b00e-b457e2175cb9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:29.090758 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835841cf-81ce-40fd-b00e-b457e2175cb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:29.092381 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835841cf-81ce-40fd-b00e-b457e2175cb9 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:29.102539 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 544/1096] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Aug 8 16:33:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:29.109688 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee592ea-d89f-4c8d-8d9b-429d5a6c9b32 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:29.142746 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:29.240587 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee592ea-d89f-4c8d-8d9b-429d5a6c9b32 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:29.282925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e3968-ed88-43d7-8073-1221eb17ef92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:29.283712 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e3968-ed88-43d7-8073-1221eb17ef92 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:29.284084 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e3968-ed88-43d7-8073-1221eb17ef92 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:29.284453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e3968-ed88-43d7-8073-1221eb17ef92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:29.297657 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e3968-ed88-43d7-8073-1221eb17ef92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:29.299422 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e3968-ed88-43d7-8073-1221eb17ef92 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:29.317295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-b17e3968-ed88-43d7-8073-1221eb17ef92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:33:29.317295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:29.317295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:29.317295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:29.317295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:29.317295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:29.317295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:29.317295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:29.317295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:29.317295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:29.317295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:29.317295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:29.317295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:33:29.317295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:33:29.317295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:33:29.317295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:33:29.318558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:29.318558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:29.318558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:29.318558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:29.318558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:29.318558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:29.318558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:29.318558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:29.318558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:29.318558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:29.318558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:29.318558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:29.318558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:29.318558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:29.318558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:29.318558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:29.318558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:33:29.320500 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:33:29.320500 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:33:29.320500 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:29.320500 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 553/1097] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:33:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:29.325424 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835841cf-81ce-40fd-b00e-b457e2175cb9 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:29.396843 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:29.452670 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835841cf-81ce-40fd-b00e-b457e2175cb9 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:29.495650 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b6019-c9b1-4228-9b31-cc6e76502b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:29.496279 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b6019-c9b1-4228-9b31-cc6e76502b92 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:29.496555 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b6019-c9b1-4228-9b31-cc6e76502b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:29.496826 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b6019-c9b1-4228-9b31-cc6e76502b92 None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:29.497496 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b6019-c9b1-4228-9b31-cc6e76502b92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:29.497794 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b6019-c9b1-4228-9b31-cc6e76502b92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:29.499289 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b6019-c9b1-4228-9b31-cc6e76502b92 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:29.515206 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 545/1098] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:29.521294 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e3968-ed88-43d7-8073-1221eb17ef92 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:29.614294 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:29.645043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e3968-ed88-43d7-8073-1221eb17ef92 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:29.687435 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351c032-5df0-46ef-b628-2377b8701c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:29.687995 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351c032-5df0-46ef-b628-2377b8701c62 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:29.688272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351c032-5df0-46ef-b628-2377b8701c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:29.688550 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351c032-5df0-46ef-b628-2377b8701c62 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:29.698896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351c032-5df0-46ef-b628-2377b8701c62 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:29.700470 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351c032-5df0-46ef-b628-2377b8701c62 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:29.713374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-6351c032-5df0-46ef-b628-2377b8701c62 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:33:29.713374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:29.713374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:29.713374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:29.713374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:29.713374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:29.713374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:29.713374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:29.713374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:29.713374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:29.713374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:29.713374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:29.713374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:33:29.713374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:33:29.713374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:33:29.713374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:33:29.715408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:29.715408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:29.715408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:33:29.715408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:33:29.715408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:33:29.715408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:33:29.715408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:29.715408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:29.715408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:29.715408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:29.715408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:29.715408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:29.715408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:29.715408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:29.715408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:29.715408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:29.715408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:29.715408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:29.717442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:29.717442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:29.717442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:33:29.717442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:33:29.717442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:33:29.717442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:33:29.717442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:33:29.717442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:33:29.717442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:33:29.717442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:29.717442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 554/1099] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Aug 8 16:33:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:33:29.721627 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b6019-c9b1-4228-9b31-cc6e76502b92 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:29.901363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b6019-c9b1-4228-9b31-cc6e76502b92 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:29.934688 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:29.957377 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf19c0c-7d3e-47d3-9162-d6d383c49082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:29.957377 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf19c0c-7d3e-47d3-9162-d6d383c49082 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:29.957377 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf19c0c-7d3e-47d3-9162-d6d383c49082 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:29.957377 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf19c0c-7d3e-47d3-9162-d6d383c49082 None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:29.958341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf19c0c-7d3e-47d3-9162-d6d383c49082 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:29.958341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf19c0c-7d3e-47d3-9162-d6d383c49082 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:29.958341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf19c0c-7d3e-47d3-9162-d6d383c49082 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:29.983676 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 546/1100] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Aug 8 16:33:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:29.989864 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351c032-5df0-46ef-b628-2377b8701c62 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:30.106494 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351c032-5df0-46ef-b628-2377b8701c62 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:30.148912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e586871-6ce1-42fe-86da-fc057c567149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:30.149540 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e586871-6ce1-42fe-86da-fc057c567149 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:30.149817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e586871-6ce1-42fe-86da-fc057c567149 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:30.150084 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e586871-6ce1-42fe-86da-fc057c567149 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:30.153335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:30.242111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e586871-6ce1-42fe-86da-fc057c567149 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:30.243531 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e586871-6ce1-42fe-86da-fc057c567149 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:30.360425 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 555/1101] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:33:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 16:33:30.366743 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf19c0c-7d3e-47d3-9162-d6d383c49082 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:30.500633 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf19c0c-7d3e-47d3-9162-d6d383c49082 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:30.544572 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5185cd3-c0e8-4b60-94be-7d10b0a563b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:30.545300 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5185cd3-c0e8-4b60-94be-7d10b0a563b4 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:30.545641 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5185cd3-c0e8-4b60-94be-7d10b0a563b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:30.545977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5185cd3-c0e8-4b60-94be-7d10b0a563b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:30.560179 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5185cd3-c0e8-4b60-94be-7d10b0a563b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:30.561669 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5185cd3-c0e8-4b60-94be-7d10b0a563b4 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:30.586528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-f5185cd3-c0e8-4b60-94be-7d10b0a563b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:33:30.586528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:30.586528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:30.586528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:30.586528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:30.586528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:30.586528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:30.586528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:30.586528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:30.586528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:30.586528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:30.586528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:30.586528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:33:30.586528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:33:30.586528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:33:30.586528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:33:30.588064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:30.588064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:30.588064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:30.588064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:30.588064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:30.588064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:30.588064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:30.588064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:30.588064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:30.588064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:30.588064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:30.588064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:30.588064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:30.588064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:30.588064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:30.588064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:30.588064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:33:30.589200 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:33:30.589200 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:33:30.589200 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:30.589200 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 547/1102] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:33:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:33:30.598307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e586871-6ce1-42fe-86da-fc057c567149 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:30.640427 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:30.647844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423da330-66d8-48df-99db-5197ae68abdd None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:30.648308 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423da330-66d8-48df-99db-5197ae68abdd None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:30.652761 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423da330-66d8-48df-99db-5197ae68abdd None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:30.652761 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 548/1103] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:33:30.657038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206eb685-02e5-448f-9c3d-1197df448b34 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:30.657322 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206eb685-02e5-448f-9c3d-1197df448b34 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:30.657589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206eb685-02e5-448f-9c3d-1197df448b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:30.676957 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-206eb685-02e5-448f-9c3d-1197df448b34 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:30.678194 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-206eb685-02e5-448f-9c3d-1197df448b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:30.712585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:30.720513 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e586871-6ce1-42fe-86da-fc057c567149 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:30.760162 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75943e-272c-4527-8399-abc2a1b13a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:30.760859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75943e-272c-4527-8399-abc2a1b13a60 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:30.761203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75943e-272c-4527-8399-abc2a1b13a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:30.761542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75943e-272c-4527-8399-abc2a1b13a60 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:30.762347 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d75943e-272c-4527-8399-abc2a1b13a60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:30.762743 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d75943e-272c-4527-8399-abc2a1b13a60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:30.764298 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d75943e-272c-4527-8399-abc2a1b13a60 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:30.777795 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 556/1104] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:30.784048 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75943e-272c-4527-8399-abc2a1b13a60 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:30.787890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 549/1105] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:33:30.814330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206eb685-02e5-448f-9c3d-1197df448b34 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:30.927912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:30.939394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75943e-272c-4527-8399-abc2a1b13a60 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:30.953549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206eb685-02e5-448f-9c3d-1197df448b34 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:30.983265 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6fb11-e514-4ab9-8784-94e544775260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:30.983966 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6fb11-e514-4ab9-8784-94e544775260 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:30.984867 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6fb11-e514-4ab9-8784-94e544775260 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:30.984867 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6fb11-e514-4ab9-8784-94e544775260 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:30.994997 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6fb11-e514-4ab9-8784-94e544775260 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:30.996524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6fb11-e514-4ab9-8784-94e544775260 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:30.997770 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35efce6c-a8ed-4f3c-b6d2-583a693d91cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:30.998677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efce6c-a8ed-4f3c-b6d2-583a693d91cd None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:30.999070 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efce6c-a8ed-4f3c-b6d2-583a693d91cd None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:30.999461 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efce6c-a8ed-4f3c-b6d2-583a693d91cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:31.000794 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35efce6c-a8ed-4f3c-b6d2-583a693d91cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:31.002451 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35efce6c-a8ed-4f3c-b6d2-583a693d91cd None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:31.004289 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 550/1106] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:33:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:31.009996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-29e6fb11-e514-4ab9-8784-94e544775260 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:33:31.009996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:31.009996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:31.009996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:31.009996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:31.009996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:31.009996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:31.009996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:31.009996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:31.009996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:31.009996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:31.009996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:31.009996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:33:31.009996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:33:31.009996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:33:31.011899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:33:31.011899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:31.011899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:31.011899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:31.011899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:31.011899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:31.011899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:31.011899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:31.011899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:31.011899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:31.011899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:31.011899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:31.011899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:31.011899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:31.011899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:31.011899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:31.011899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:31.013630 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:33:31.013630 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:33:31.013630 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:33:31.013630 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:33:31.013630 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:33:31.013630 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:33:31.013630 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:33:31.013630 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:31.014484 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35efce6c-a8ed-4f3c-b6d2-583a693d91cd None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:31.019107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 557/1107] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:33:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:31.021503 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6fb11-e514-4ab9-8784-94e544775260 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:31.052878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:31.160170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35efce6c-a8ed-4f3c-b6d2-583a693d91cd None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:31.165650 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6fb11-e514-4ab9-8784-94e544775260 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:31.217700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b85b0-20b0-4da5-b48c-ff2bbd9c0bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:31.218686 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b85b0-20b0-4da5-b48c-ff2bbd9c0bbe None admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:31.219109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b85b0-20b0-4da5-b48c-ff2bbd9c0bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:31.219560 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b85b0-20b0-4da5-b48c-ff2bbd9c0bbe None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:31.220673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b85b0-20b0-4da5-b48c-ff2bbd9c0bbe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:31.223335 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b85b0-20b0-4da5-b48c-ff2bbd9c0bbe None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:31.223889 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61805ab-08d9-4021-b61e-8c5218c89262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:31.224843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61805ab-08d9-4021-b61e-8c5218c89262 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:31.225288 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61805ab-08d9-4021-b61e-8c5218c89262 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:31.225724 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61805ab-08d9-4021-b61e-8c5218c89262 None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:31.226984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61805ab-08d9-4021-b61e-8c5218c89262 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:31.227477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61805ab-08d9-4021-b61e-8c5218c89262 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:31.229939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61805ab-08d9-4021-b61e-8c5218c89262 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:31.248915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 558/1108] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:33:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:31.255469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61805ab-08d9-4021-b61e-8c5218c89262 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:31.273612 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:31.319206 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 551/1109] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Aug 8 16:33:31 2022] POST /identity/v3/users => generated 286 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:33:31.375450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61805ab-08d9-4021-b61e-8c5218c89262 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:31.422597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea97e16-b73e-4b41-abc9-ac85d5fd0365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:31.423318 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea97e16-b73e-4b41-abc9-ac85d5fd0365 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:31.423650 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea97e16-b73e-4b41-abc9-ac85d5fd0365 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:31.423986 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea97e16-b73e-4b41-abc9-ac85d5fd0365 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:31.424832 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea97e16-b73e-4b41-abc9-ac85d5fd0365 None admin] RBAC: Adding query filter params (role.id=16940dcad5d247d0b43da26eb2b69cbc, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=9daa7489bced467984d39e8b8ac49d13) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:31.425199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea97e16-b73e-4b41-abc9-ac85d5fd0365 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:31.426794 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea97e16-b73e-4b41-abc9-ac85d5fd0365 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:31.435050 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 559/1110] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:33:31 2022] GET /identity/v3/role_assignments?user.id=9daa7489bced467984d39e8b8ac49d13&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=16940dcad5d247d0b43da26eb2b69cbc => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:33.715830 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bf11e-ea28-40b6-9794-abeb498e3ef8 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:33.716458 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bf11e-ea28-40b6-9794-abeb498e3ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:33.716458 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bf11e-ea28-40b6-9794-abeb498e3ef8 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:33.717978 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 552/1111] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:33:33.723840 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c576c7-946e-4605-aff6-dde8c7a0d53a None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:33.724373 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c576c7-946e-4605-aff6-dde8c7a0d53a None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:33.724546 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c576c7-946e-4605-aff6-dde8c7a0d53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:33.743799 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-15c576c7-946e-4605-aff6-dde8c7a0d53a None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:33.745026 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-15c576c7-946e-4605-aff6-dde8c7a0d53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:33.815506 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 560/1112] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:33:33.860188 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043bf11e-ea28-40b6-9794-abeb498e3ef8 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:33.955179 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:33.975309 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043bf11e-ea28-40b6-9794-abeb498e3ef8 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:34.011515 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f7591-0529-4ff4-bef9-151bb9f7347a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:34.012143 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f7591-0529-4ff4-bef9-151bb9f7347a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:34.012413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f7591-0529-4ff4-bef9-151bb9f7347a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:34.012674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f7591-0529-4ff4-bef9-151bb9f7347a None admin] PATH_INFO: `/v3/roles/fd2599a3f408404599ecea94959efc4a` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:34.014060 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f7591-0529-4ff4-bef9-151bb9f7347a None admin] RBAC: Authorizing `identity:get_role(role_id=fd2599a3f408404599ecea94959efc4a)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:34.015532 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f7591-0529-4ff4-bef9-151bb9f7347a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:34.016876 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 553/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:33:33 2022] GET /identity/v3/roles/fd2599a3f408404599ecea94959efc4a => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:34.023063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c576c7-946e-4605-aff6-dde8c7a0d53a None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:34.034220 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:34.127108 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c576c7-946e-4605-aff6-dde8c7a0d53a None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:34.163818 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b262395-3a12-4a83-9469-d32176d95999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:34.164464 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b262395-3a12-4a83-9469-d32176d95999 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:34.164741 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b262395-3a12-4a83-9469-d32176d95999 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:34.165007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b262395-3a12-4a83-9469-d32176d95999 None admin] PATH_INFO: `/v3/users/7880a463812d4b5a955a47a5338b1c5d` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:34.179839 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b262395-3a12-4a83-9469-d32176d95999 None admin] RBAC: Authorizing `identity:get_user(user_id=7880a463812d4b5a955a47a5338b1c5d)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:34.181280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b262395-3a12-4a83-9469-d32176d95999 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:34.182998 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 561/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:33:34 2022] GET /identity/v3/users/7880a463812d4b5a955a47a5338b1c5d => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:34.188857 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f7591-0529-4ff4-bef9-151bb9f7347a None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:34.291452 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f7591-0529-4ff4-bef9-151bb9f7347a None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:34.325362 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c15ca-cb91-4ee5-bb9e-278b6532c5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:34.325966 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c15ca-cb91-4ee5-bb9e-278b6532c5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:34.326240 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c15ca-cb91-4ee5-bb9e-278b6532c5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:34.326504 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c15ca-cb91-4ee5-bb9e-278b6532c5c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:34.346020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:34.397070 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0c15ca-cb91-4ee5-bb9e-278b6532c5c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:34.399384 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0c15ca-cb91-4ee5-bb9e-278b6532c5c3 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:34.499387 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 554/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:33:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:33:34.506109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b262395-3a12-4a83-9469-d32176d95999 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:34.579638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:34.615061 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b262395-3a12-4a83-9469-d32176d95999 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:34.652087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f7497-bfad-45d4-8975-864e4aa33445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:34.652763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f7497-bfad-45d4-8975-864e4aa33445 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:34.653337 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f7497-bfad-45d4-8975-864e4aa33445 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:34.653652 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f7497-bfad-45d4-8975-864e4aa33445 None admin] PATH_INFO: `/v3/projects/5f0fe56f23d54e67a6522dce3ed8c46a` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:34.654955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f7497-bfad-45d4-8975-864e4aa33445 None admin] RBAC: Authorizing `identity:get_project(project_id=5f0fe56f23d54e67a6522dce3ed8c46a)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:34.656448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f7497-bfad-45d4-8975-864e4aa33445 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:34.658126 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 562/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:33:34 2022] GET /identity/v3/projects/5f0fe56f23d54e67a6522dce3ed8c46a => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:34.664610 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c15ca-cb91-4ee5-bb9e-278b6532c5c3 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:34.721550 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:34.775298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c15ca-cb91-4ee5-bb9e-278b6532c5c3 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:34.809300 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6882d-f3ef-42bf-8553-b8c50a52b6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:34.809898 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6882d-f3ef-42bf-8553-b8c50a52b6ca None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:34.810169 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6882d-f3ef-42bf-8553-b8c50a52b6ca None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:34.810429 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6882d-f3ef-42bf-8553-b8c50a52b6ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:34.811171 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6882d-f3ef-42bf-8553-b8c50a52b6ca None admin] RBAC: Adding query filter params (role.id=fd2599a3f408404599ecea94959efc4a, scope.project.id=5f0fe56f23d54e67a6522dce3ed8c46a, user.id=7880a463812d4b5a955a47a5338b1c5d) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:34.811469 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6882d-f3ef-42bf-8553-b8c50a52b6ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:34.812881 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6882d-f3ef-42bf-8553-b8c50a52b6ca None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:34.818539 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 555/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:33:34 2022] GET /identity/v3/role_assignments?user.id=7880a463812d4b5a955a47a5338b1c5d&scope.project.id=5f0fe56f23d54e67a6522dce3ed8c46a&role.id=fd2599a3f408404599ecea94959efc4a => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:36.933154 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b9644-6ff1-462a-aea9-7eba18093dda None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:36.933745 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b9644-6ff1-462a-aea9-7eba18093dda None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:36.933745 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b9644-6ff1-462a-aea9-7eba18093dda None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:36.935108 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 563/1118] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:33:36.940998 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cb570-1ea3-4450-8509-ec141dc0d86e None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:36.941543 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cb570-1ea3-4450-8509-ec141dc0d86e None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:36.941543 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cb570-1ea3-4450-8509-ec141dc0d86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:36.961934 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-9d3cb570-1ea3-4450-8509-ec141dc0d86e None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:36.963191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-9d3cb570-1ea3-4450-8509-ec141dc0d86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:37.040089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 556/1119] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:33:37.063225 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b9644-6ff1-462a-aea9-7eba18093dda None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:37.176265 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b9644-6ff1-462a-aea9-7eba18093dda None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:37.213695 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421534b-be1e-4f45-ba34-1b4386d23f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:37.214255 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421534b-be1e-4f45-ba34-1b4386d23f37 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:37.214532 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421534b-be1e-4f45-ba34-1b4386d23f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:37.214839 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421534b-be1e-4f45-ba34-1b4386d23f37 None admin] PATH_INFO: `/v3/roles/fd2599a3f408404599ecea94959efc4a` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:37.216268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421534b-be1e-4f45-ba34-1b4386d23f37 None admin] RBAC: Authorizing `identity:get_role(role_id=fd2599a3f408404599ecea94959efc4a)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:37.217895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421534b-be1e-4f45-ba34-1b4386d23f37 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:37.219293 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 564/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:33:37 2022] GET /identity/v3/roles/fd2599a3f408404599ecea94959efc4a => generated 223 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:37.228985 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cb570-1ea3-4450-8509-ec141dc0d86e None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:37.275594 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:37.332842 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cb570-1ea3-4450-8509-ec141dc0d86e None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:37.369086 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ace23c-7cc4-426b-ab77-b51a2df17fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:37.369770 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ace23c-7cc4-426b-ab77-b51a2df17fbd None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:37.370089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ace23c-7cc4-426b-ab77-b51a2df17fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:37.370400 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ace23c-7cc4-426b-ab77-b51a2df17fbd None admin] PATH_INFO: `/v3/users/7880a463812d4b5a955a47a5338b1c5d` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:37.371608 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ace23c-7cc4-426b-ab77-b51a2df17fbd None admin] RBAC: Authorizing `identity:get_user(user_id=7880a463812d4b5a955a47a5338b1c5d)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:37.373092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ace23c-7cc4-426b-ab77-b51a2df17fbd None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:37.374813 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 557/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:33:37 2022] GET /identity/v3/users/7880a463812d4b5a955a47a5338b1c5d => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:37.381057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421534b-be1e-4f45-ba34-1b4386d23f37 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:37.384289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:37.536609 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421534b-be1e-4f45-ba34-1b4386d23f37 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:37.596790 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c422ee-1481-4d5e-b0ac-ac52edb519ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:37.597862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c422ee-1481-4d5e-b0ac-ac52edb519ab None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:37.598361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c422ee-1481-4d5e-b0ac-ac52edb519ab None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:37.599144 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c422ee-1481-4d5e-b0ac-ac52edb519ab None admin] PATH_INFO: `/v3/projects/5f0fe56f23d54e67a6522dce3ed8c46a` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:37.601578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c422ee-1481-4d5e-b0ac-ac52edb519ab None admin] RBAC: Authorizing `identity:get_project(project_id=5f0fe56f23d54e67a6522dce3ed8c46a)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:37.604283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c422ee-1481-4d5e-b0ac-ac52edb519ab None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:37.607423 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 565/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:33:37 2022] GET /identity/v3/projects/5f0fe56f23d54e67a6522dce3ed8c46a => generated 311 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:37.613775 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ace23c-7cc4-426b-ab77-b51a2df17fbd None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:37.720393 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ace23c-7cc4-426b-ab77-b51a2df17fbd None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:37.736540 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:37.759132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf5d79-5513-4c3e-8392-7351fe56c82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:37.759846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf5d79-5513-4c3e-8392-7351fe56c82a None admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:37.760185 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf5d79-5513-4c3e-8392-7351fe56c82a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:37.760498 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf5d79-5513-4c3e-8392-7351fe56c82a None admin] PATH_INFO: `/v3/projects/5f0fe56f23d54e67a6522dce3ed8c46a/users/7880a463812d4b5a955a47a5338b1c5d/roles/fd2599a3f408404599ecea94959efc4a` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:37.762924 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf5d79-5513-4c3e-8392-7351fe56c82a None admin] RBAC: Authorizing `identity:create_grant(project_id=5f0fe56f23d54e67a6522dce3ed8c46a, user_id=7880a463812d4b5a955a47a5338b1c5d, role_id=fd2599a3f408404599ecea94959efc4a)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:37.764415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf5d79-5513-4c3e-8392-7351fe56c82a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:37.828668 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 558/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:33:37 2022] PUT /identity/v3/projects/5f0fe56f23d54e67a6522dce3ed8c46a/users/7880a463812d4b5a955a47a5338b1c5d/roles/fd2599a3f408404599ecea94959efc4a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:33:39.911515 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549c9f7-c70f-4c01-88b2-d008f8ae711a None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:39.912124 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549c9f7-c70f-4c01-88b2-d008f8ae711a None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:39.912124 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549c9f7-c70f-4c01-88b2-d008f8ae711a None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:39.913890 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 566/1124] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:33:39.920419 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343f5af-3860-4fa6-9c0c-e3f3311ea76f None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:39.920965 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343f5af-3860-4fa6-9c0c-e3f3311ea76f None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:39.921250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343f5af-3860-4fa6-9c0c-e3f3311ea76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:39.942800 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-a343f5af-3860-4fa6-9c0c-e3f3311ea76f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:39.944122 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-a343f5af-3860-4fa6-9c0c-e3f3311ea76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:40.014904 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 559/1125] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:33:40.039612 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549c9f7-c70f-4c01-88b2-d008f8ae711a None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:40.115529 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:40.148816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549c9f7-c70f-4c01-88b2-d008f8ae711a None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:40.187070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d09914-24e7-4e04-9806-55680809c4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:40.187625 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d09914-24e7-4e04-9806-55680809c4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:40.187903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d09914-24e7-4e04-9806-55680809c4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:40.188177 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d09914-24e7-4e04-9806-55680809c4d3 None admin] PATH_INFO: `/v3/roles/fd2599a3f408404599ecea94959efc4a` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:40.189582 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d09914-24e7-4e04-9806-55680809c4d3 None admin] RBAC: Authorizing `identity:get_role(role_id=fd2599a3f408404599ecea94959efc4a)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:40.191085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d09914-24e7-4e04-9806-55680809c4d3 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:40.193258 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 567/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:33:40 2022] GET /identity/v3/roles/fd2599a3f408404599ecea94959efc4a => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:40.199435 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343f5af-3860-4fa6-9c0c-e3f3311ea76f None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:40.303769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343f5af-3860-4fa6-9c0c-e3f3311ea76f None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:40.305436 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:40.339849 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae10164-d4ec-4c27-a17e-104ea8aac40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:40.340680 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae10164-d4ec-4c27-a17e-104ea8aac40a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:40.341510 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae10164-d4ec-4c27-a17e-104ea8aac40a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:40.342188 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae10164-d4ec-4c27-a17e-104ea8aac40a None admin] PATH_INFO: `/v3/users/7880a463812d4b5a955a47a5338b1c5d` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:40.343623 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae10164-d4ec-4c27-a17e-104ea8aac40a None admin] RBAC: Authorizing `identity:get_user(user_id=7880a463812d4b5a955a47a5338b1c5d)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:40.345223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae10164-d4ec-4c27-a17e-104ea8aac40a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:40.347005 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 560/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:33:40 2022] GET /identity/v3/users/7880a463812d4b5a955a47a5338b1c5d => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:40.354794 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d09914-24e7-4e04-9806-55680809c4d3 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:40.459482 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d09914-24e7-4e04-9806-55680809c4d3 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:40.496143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941fd08-4c0b-4af1-98e2-1b7256a29ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:40.496752 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941fd08-4c0b-4af1-98e2-1b7256a29ffc None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:40.497025 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941fd08-4c0b-4af1-98e2-1b7256a29ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:40.497289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941fd08-4c0b-4af1-98e2-1b7256a29ffc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:40.506485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:40.572483 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941fd08-4c0b-4af1-98e2-1b7256a29ffc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:40.573928 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941fd08-4c0b-4af1-98e2-1b7256a29ffc None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:40.682965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 568/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:33:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:33:40.689018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae10164-d4ec-4c27-a17e-104ea8aac40a None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:40.796140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae10164-d4ec-4c27-a17e-104ea8aac40a None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:40.831008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07fe5b-301e-4309-8ec7-843024d50b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:40.831631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07fe5b-301e-4309-8ec7-843024d50b8a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:40.831904 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07fe5b-301e-4309-8ec7-843024d50b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:40.832167 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07fe5b-301e-4309-8ec7-843024d50b8a None admin] PATH_INFO: `/v3/projects/5f0fe56f23d54e67a6522dce3ed8c46a` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:40.833274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07fe5b-301e-4309-8ec7-843024d50b8a None admin] RBAC: Authorizing `identity:get_project(project_id=5f0fe56f23d54e67a6522dce3ed8c46a)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:40.834740 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07fe5b-301e-4309-8ec7-843024d50b8a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:40.836957 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 561/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:33:40 2022] GET /identity/v3/projects/5f0fe56f23d54e67a6522dce3ed8c46a => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:40.843646 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941fd08-4c0b-4af1-98e2-1b7256a29ffc None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:40.886712 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:40.949183 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941fd08-4c0b-4af1-98e2-1b7256a29ffc None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:40.984699 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404f73c-fcb7-41d2-9003-affe5e255dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:40.985307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404f73c-fcb7-41d2-9003-affe5e255dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:40.985587 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404f73c-fcb7-41d2-9003-affe5e255dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:40.985858 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404f73c-fcb7-41d2-9003-affe5e255dc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:40.986597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404f73c-fcb7-41d2-9003-affe5e255dc9 None admin] RBAC: Adding query filter params (role.id=fd2599a3f408404599ecea94959efc4a, scope.project.id=5f0fe56f23d54e67a6522dce3ed8c46a, user.id=7880a463812d4b5a955a47a5338b1c5d) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:40.986915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404f73c-fcb7-41d2-9003-affe5e255dc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:40.988344 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404f73c-fcb7-41d2-9003-affe5e255dc9 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:40.994925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 569/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:33:40 2022] GET /identity/v3/role_assignments?user.id=7880a463812d4b5a955a47a5338b1c5d&scope.project.id=5f0fe56f23d54e67a6522dce3ed8c46a&role.id=fd2599a3f408404599ecea94959efc4a => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:43.118363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c96b3b-6e94-442b-ba09-aefe3ae712a3 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:43.118988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c96b3b-6e94-442b-ba09-aefe3ae712a3 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:43.118988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c96b3b-6e94-442b-ba09-aefe3ae712a3 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:43.120277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 562/1131] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:33:43.126224 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59306961-8c5e-49d5-b159-04a261a0ace1 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:43.126853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59306961-8c5e-49d5-b159-04a261a0ace1 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:43.126853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59306961-8c5e-49d5-b159-04a261a0ace1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:43.145091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-59306961-8c5e-49d5-b159-04a261a0ace1 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:43.146295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-59306961-8c5e-49d5-b159-04a261a0ace1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:43.242096 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 570/1132] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:33:43.267128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c96b3b-6e94-442b-ba09-aefe3ae712a3 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:43.312943 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:43.379293 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c96b3b-6e94-442b-ba09-aefe3ae712a3 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:43.421343 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038e2e9-b031-4141-8799-3a73b2efe191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:43.421976 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038e2e9-b031-4141-8799-3a73b2efe191 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:43.422252 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038e2e9-b031-4141-8799-3a73b2efe191 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:43.422518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038e2e9-b031-4141-8799-3a73b2efe191 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:43.423711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038e2e9-b031-4141-8799-3a73b2efe191 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:43.425211 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038e2e9-b031-4141-8799-3a73b2efe191 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:43.426943 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 563/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:33:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:43.433525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59306961-8c5e-49d5-b159-04a261a0ace1 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:43.484552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:43.562991 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59306961-8c5e-49d5-b159-04a261a0ace1 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:43.603240 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c711b-f5be-4e66-86f4-487852830305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:43.603893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c711b-f5be-4e66-86f4-487852830305 None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:43.604210 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c711b-f5be-4e66-86f4-487852830305 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:43.604528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c711b-f5be-4e66-86f4-487852830305 None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:43.605224 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c711b-f5be-4e66-86f4-487852830305 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:43.606739 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c711b-f5be-4e66-86f4-487852830305 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:43.722539 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 571/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Aug 8 16:33:43 2022] POST /identity/v3/projects => generated 311 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:33:45.943137 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941141ed-27fa-4ef8-8eaf-bc5a8d9514f9 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:45.944014 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941141ed-27fa-4ef8-8eaf-bc5a8d9514f9 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:45.944014 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941141ed-27fa-4ef8-8eaf-bc5a8d9514f9 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:45.945697 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 564/1135] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:33:45.951088 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcc639-5dbf-4ba9-b701-1dac0a151593 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:45.951621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcc639-5dbf-4ba9-b701-1dac0a151593 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:45.951751 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcc639-5dbf-4ba9-b701-1dac0a151593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:45.970924 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-f7dcc639-5dbf-4ba9-b701-1dac0a151593 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:45.972140 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f7dcc639-5dbf-4ba9-b701-1dac0a151593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:46.043421 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 572/1136] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:33:46.068606 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941141ed-27fa-4ef8-8eaf-bc5a8d9514f9 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:46.181574 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941141ed-27fa-4ef8-8eaf-bc5a8d9514f9 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:46.190989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:46.219840 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515732d-0bcf-46ac-a653-8b3c79003c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:46.220480 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515732d-0bcf-46ac-a653-8b3c79003c97 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:46.220758 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515732d-0bcf-46ac-a653-8b3c79003c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:46.221026 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515732d-0bcf-46ac-a653-8b3c79003c97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:46.222376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4515732d-0bcf-46ac-a653-8b3c79003c97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:46.224508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4515732d-0bcf-46ac-a653-8b3c79003c97 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:46.226734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 565/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:33:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:46.232856 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dcc639-5dbf-4ba9-b701-1dac0a151593 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:46.251593 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:46.342958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dcc639-5dbf-4ba9-b701-1dac0a151593 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:46.378984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a518fc-c34f-4b9e-87d4-5a6cf111ed46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:46.379599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a518fc-c34f-4b9e-87d4-5a6cf111ed46 None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:46.379877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a518fc-c34f-4b9e-87d4-5a6cf111ed46 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:46.380155 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a518fc-c34f-4b9e-87d4-5a6cf111ed46 None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:46.380824 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a518fc-c34f-4b9e-87d4-5a6cf111ed46 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:46.382850 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a518fc-c34f-4b9e-87d4-5a6cf111ed46 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:46.497185 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 573/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Aug 8 16:33:46 2022] POST /identity/v3/users => generated 286 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:33:48.651502 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43779c-ac16-4b56-9ae8-fc59692d62b5 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:48.652086 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43779c-ac16-4b56-9ae8-fc59692d62b5 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:48.652086 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43779c-ac16-4b56-9ae8-fc59692d62b5 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:48.653378 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 566/1139] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:33:48.659245 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a5d6e-689a-4b01-95e3-8e99aa3bf8c2 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:48.660016 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a5d6e-689a-4b01-95e3-8e99aa3bf8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:48.660263 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a5d6e-689a-4b01-95e3-8e99aa3bf8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:48.680049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-b03a5d6e-689a-4b01-95e3-8e99aa3bf8c2 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:48.681257 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b03a5d6e-689a-4b01-95e3-8e99aa3bf8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:48.753629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:48.759025 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 574/1140] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:33:48.782371 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43779c-ac16-4b56-9ae8-fc59692d62b5 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:48.891002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43779c-ac16-4b56-9ae8-fc59692d62b5 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:48.927018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9b93f-144a-4cc0-ad8e-4e2e98788a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:48.927582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9b93f-144a-4cc0-ad8e-4e2e98788a4a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:48.927852 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9b93f-144a-4cc0-ad8e-4e2e98788a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:48.928112 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9b93f-144a-4cc0-ad8e-4e2e98788a4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:48.939751 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e9b93f-144a-4cc0-ad8e-4e2e98788a4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:33:48.944989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-44e9b93f-144a-4cc0-ad8e-4e2e98788a4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:33:48.944989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:48.944989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:48.944989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:48.944989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:48.944989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:48.944989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:48.944989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:48.944989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:48.944989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:48.944989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:48.944989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:48.944989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:33:48.944989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:33:48.944989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:33:48.944989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:33:48.946283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:33:48.946283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:33:48.946283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:33:48.946283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:33:48.946283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:48.946283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:48.946283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:48.946283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:48.946283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:48.946283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:48.946283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:48.946283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:48.946283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:48.946283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:48.946283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:48.946283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:48.947482 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:48.947482 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:48.947482 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:48.947482 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:48.947482 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:33:48.947482 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:33:48.947482 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:33:48.947482 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:33:48.947482 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:33:48.947482 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:33:48.947482 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:33:48.947482 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:48.947482 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 567/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:33:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:48.955199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a5d6e-689a-4b01-95e3-8e99aa3bf8c2 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:49.074556 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a5d6e-689a-4b01-95e3-8e99aa3bf8c2 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:49.118545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f58027-660e-4f50-8f12-2d43991efee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:49.119196 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f58027-660e-4f50-8f12-2d43991efee6 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:49.119472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f58027-660e-4f50-8f12-2d43991efee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:49.119741 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f58027-660e-4f50-8f12-2d43991efee6 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:49.120424 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f58027-660e-4f50-8f12-2d43991efee6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:49.120723 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f58027-660e-4f50-8f12-2d43991efee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:49.122122 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f58027-660e-4f50-8f12-2d43991efee6 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:49.131863 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 575/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:33:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:49.137938 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9b93f-144a-4cc0-ad8e-4e2e98788a4a None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:49.179928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:49.252225 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9b93f-144a-4cc0-ad8e-4e2e98788a4a None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:49.290681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b7ed7-d628-4e4d-ba0b-3fa117f11ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:49.291312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b7ed7-d628-4e4d-ba0b-3fa117f11ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:49.291585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b7ed7-d628-4e4d-ba0b-3fa117f11ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:49.291849 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b7ed7-d628-4e4d-ba0b-3fa117f11ed5 None admin] PATH_INFO: `/v3/users/085c647729604e268464241be65a66f0` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:49.307477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b7ed7-d628-4e4d-ba0b-3fa117f11ed5 None admin] RBAC: Authorizing `identity:get_user(user_id=085c647729604e268464241be65a66f0)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:49.309083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b7ed7-d628-4e4d-ba0b-3fa117f11ed5 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:49.310870 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 568/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:33:49 2022] GET /identity/v3/users/085c647729604e268464241be65a66f0 => generated 286 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:49.317516 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f58027-660e-4f50-8f12-2d43991efee6 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:49.394926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:49.440655 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f58027-660e-4f50-8f12-2d43991efee6 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:49.478618 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2ebb8-c273-4da9-a9cc-9c347d5dc5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:49.479248 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2ebb8-c273-4da9-a9cc-9c347d5dc5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:49.479526 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2ebb8-c273-4da9-a9cc-9c347d5dc5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:49.479795 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2ebb8-c273-4da9-a9cc-9c347d5dc5c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:49.554415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c2ebb8-c273-4da9-a9cc-9c347d5dc5c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:49.556032 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c2ebb8-c273-4da9-a9cc-9c347d5dc5c0 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:49.623936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:49.665976 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 576/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:33:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:33:49.673061 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b7ed7-d628-4e4d-ba0b-3fa117f11ed5 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:49.771407 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:49.780015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b7ed7-d628-4e4d-ba0b-3fa117f11ed5 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:49.818786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db221b2-832c-4ba9-ab6e-88a89865f737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:49.820197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db221b2-832c-4ba9-ab6e-88a89865f737 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:49.820484 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db221b2-832c-4ba9-ab6e-88a89865f737 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:49.820755 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db221b2-832c-4ba9-ab6e-88a89865f737 None admin] PATH_INFO: `/v3/projects/1d0eb32ba0494f7c914511315fdaa594` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:49.821937 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db221b2-832c-4ba9-ab6e-88a89865f737 None admin] RBAC: Authorizing `identity:get_project(project_id=1d0eb32ba0494f7c914511315fdaa594)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:49.823410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db221b2-832c-4ba9-ab6e-88a89865f737 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:49.825073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 569/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:33:49 2022] GET /identity/v3/projects/1d0eb32ba0494f7c914511315fdaa594 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:49.831432 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2ebb8-c273-4da9-a9cc-9c347d5dc5c0 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:49.946732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2ebb8-c273-4da9-a9cc-9c347d5dc5c0 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:49.996866 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51822ce8-4593-4397-b912-639348d1488c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:49.997467 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51822ce8-4593-4397-b912-639348d1488c None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:49.997734 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51822ce8-4593-4397-b912-639348d1488c None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:49.998007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51822ce8-4593-4397-b912-639348d1488c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:49.998754 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51822ce8-4593-4397-b912-639348d1488c None admin] RBAC: Adding query filter params (role.id=e578ef831127463d9b48cc89fe20329c, scope.project.id=1d0eb32ba0494f7c914511315fdaa594, user.id=085c647729604e268464241be65a66f0) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:49.999056 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51822ce8-4593-4397-b912-639348d1488c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:50.000477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51822ce8-4593-4397-b912-639348d1488c None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:50.006143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 577/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:33:49 2022] GET /identity/v3/role_assignments?user.id=085c647729604e268464241be65a66f0&scope.project.id=1d0eb32ba0494f7c914511315fdaa594&role.id=e578ef831127463d9b48cc89fe20329c => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:52.086933 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92be5b-e582-43d0-8908-a0a1bfbde2ef None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:52.087521 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92be5b-e582-43d0-8908-a0a1bfbde2ef None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:52.087521 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92be5b-e582-43d0-8908-a0a1bfbde2ef None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:52.088850 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 570/1147] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:33:52.094882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ddb34-f8db-4ed2-aee4-e55c8d88bbcd None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:52.095469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ddb34-f8db-4ed2-aee4-e55c8d88bbcd None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:52.095469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ddb34-f8db-4ed2-aee4-e55c8d88bbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:52.114047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-b03ddb34-f8db-4ed2-aee4-e55c8d88bbcd None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:52.115319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b03ddb34-f8db-4ed2-aee4-e55c8d88bbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:52.141105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:52.198688 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 578/1148] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:33:52.222605 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92be5b-e582-43d0-8908-a0a1bfbde2ef None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:52.345151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92be5b-e582-43d0-8908-a0a1bfbde2ef None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:52.383247 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6305f16-c74a-4ca1-801d-06abfc25e2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:52.383904 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6305f16-c74a-4ca1-801d-06abfc25e2e7 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:52.384182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6305f16-c74a-4ca1-801d-06abfc25e2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:52.384453 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6305f16-c74a-4ca1-801d-06abfc25e2e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:52.396822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6305f16-c74a-4ca1-801d-06abfc25e2e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:33:52.402447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-c6305f16-c74a-4ca1-801d-06abfc25e2e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:33:52.402447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:52.402447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:52.402447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:52.402447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:52.402447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:52.402447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:52.402447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:52.402447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:52.402447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:52.402447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:52.402447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:52.402447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:33:52.402447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:33:52.402447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:33:52.402447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:33:52.404353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:33:52.404353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:33:52.404353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:33:52.404353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:33:52.404353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:52.404353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:52.404353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:52.404353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:52.404353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:52.404353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:52.404353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:52.404353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:52.404353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:52.404353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:52.404353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:52.404353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:52.406204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:52.406204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:52.406204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:52.406204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:52.406204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:33:52.406204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:33:52.406204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:33:52.406204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:33:52.406204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:33:52.406204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:33:52.406204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:33:52.406204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:52.406204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 571/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:33:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:52.411770 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ddb34-f8db-4ed2-aee4-e55c8d88bbcd None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:52.527689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ddb34-f8db-4ed2-aee4-e55c8d88bbcd None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:52.567736 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a49cbd-8ee6-4a90-aadb-79b9790fecce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:52.568380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a49cbd-8ee6-4a90-aadb-79b9790fecce None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:52.568661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a49cbd-8ee6-4a90-aadb-79b9790fecce None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:52.568934 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a49cbd-8ee6-4a90-aadb-79b9790fecce None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:52.570208 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a49cbd-8ee6-4a90-aadb-79b9790fecce None admin] RBAC: Adding query filter params (name=admin) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:52.570718 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a49cbd-8ee6-4a90-aadb-79b9790fecce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:52.572160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a49cbd-8ee6-4a90-aadb-79b9790fecce None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:52.581346 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 579/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:33:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:33:52.588550 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6305f16-c74a-4ca1-801d-06abfc25e2e7 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:52.619857 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:52.708626 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6305f16-c74a-4ca1-801d-06abfc25e2e7 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:52.747919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27ced8-7db7-4c27-8023-56a04361ceaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:52.748559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27ced8-7db7-4c27-8023-56a04361ceaa None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:52.748840 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27ced8-7db7-4c27-8023-56a04361ceaa None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:52.749111 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27ced8-7db7-4c27-8023-56a04361ceaa None admin] PATH_INFO: `/v3/users/085c647729604e268464241be65a66f0` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:52.750320 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27ced8-7db7-4c27-8023-56a04361ceaa None admin] RBAC: Authorizing `identity:get_user(user_id=085c647729604e268464241be65a66f0)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:52.751826 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27ced8-7db7-4c27-8023-56a04361ceaa None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:52.753527 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 572/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:33:52 2022] GET /identity/v3/users/085c647729604e268464241be65a66f0 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:52.763813 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a49cbd-8ee6-4a90-aadb-79b9790fecce None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:52.797546 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:52.899973 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a49cbd-8ee6-4a90-aadb-79b9790fecce None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:52.938169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dadff80-e625-40f7-acde-5a22c2b3d246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:52.938847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadff80-e625-40f7-acde-5a22c2b3d246 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:52.939123 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadff80-e625-40f7-acde-5a22c2b3d246 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:52.939390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadff80-e625-40f7-acde-5a22c2b3d246 None admin] PATH_INFO: `/v3/projects/1d0eb32ba0494f7c914511315fdaa594` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:52.940545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dadff80-e625-40f7-acde-5a22c2b3d246 None admin] RBAC: Authorizing `identity:get_project(project_id=1d0eb32ba0494f7c914511315fdaa594)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:52.941974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dadff80-e625-40f7-acde-5a22c2b3d246 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:52.943707 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 580/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:33:52 2022] GET /identity/v3/projects/1d0eb32ba0494f7c914511315fdaa594 => generated 311 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:52.954666 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27ced8-7db7-4c27-8023-56a04361ceaa None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:53.110859 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:53.132679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27ced8-7db7-4c27-8023-56a04361ceaa None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:53.181101 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118316c9-ac36-44bd-b11b-03f8b15cbdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:53.181768 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118316c9-ac36-44bd-b11b-03f8b15cbdc1 None admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:53.182044 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118316c9-ac36-44bd-b11b-03f8b15cbdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:53.182315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118316c9-ac36-44bd-b11b-03f8b15cbdc1 None admin] PATH_INFO: `/v3/projects/1d0eb32ba0494f7c914511315fdaa594/users/085c647729604e268464241be65a66f0/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:53.184384 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118316c9-ac36-44bd-b11b-03f8b15cbdc1 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d0eb32ba0494f7c914511315fdaa594, user_id=085c647729604e268464241be65a66f0, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:53.185841 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118316c9-ac36-44bd-b11b-03f8b15cbdc1 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:53.283537 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 573/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:33:52 2022] PUT /identity/v3/projects/1d0eb32ba0494f7c914511315fdaa594/users/085c647729604e268464241be65a66f0/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:33:55.372682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5b973-50fc-437a-98f5-3cfc8538c5a3 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:55.373559 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5b973-50fc-437a-98f5-3cfc8538c5a3 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:55.373559 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5b973-50fc-437a-98f5-3cfc8538c5a3 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:55.374930 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 581/1154] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:33:55.380938 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e71fa-1b71-4ed1-a042-068fba706df0 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:55.381507 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e71fa-1b71-4ed1-a042-068fba706df0 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:55.381507 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e71fa-1b71-4ed1-a042-068fba706df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:55.401180 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-893e71fa-1b71-4ed1-a042-068fba706df0 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:55.402421 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-893e71fa-1b71-4ed1-a042-068fba706df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:55.484760 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 574/1155] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:33:55.510796 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5b973-50fc-437a-98f5-3cfc8538c5a3 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:55.613363 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:55.632821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5b973-50fc-437a-98f5-3cfc8538c5a3 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:55.683331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73009a94-3ab4-4d39-b018-22b2ca6c5128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:55.683991 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73009a94-3ab4-4d39-b018-22b2ca6c5128 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:55.684267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73009a94-3ab4-4d39-b018-22b2ca6c5128 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:55.684535 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73009a94-3ab4-4d39-b018-22b2ca6c5128 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:55.698055 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73009a94-3ab4-4d39-b018-22b2ca6c5128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:33:55.703498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-73009a94-3ab4-4d39-b018-22b2ca6c5128 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:33:55.703498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:55.703498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:55.703498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:55.703498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:55.703498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:55.703498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:55.703498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:55.703498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:55.703498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:55.703498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:55.703498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:55.703498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:33:55.703498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:33:55.703498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:33:55.703498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:33:55.705703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:33:55.705703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:33:55.705703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:33:55.705703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:33:55.705703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:55.705703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:55.705703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:55.705703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:55.705703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:55.705703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:55.705703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:55.705703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:55.705703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:55.705703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:55.705703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:55.705703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:55.707937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:55.707937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:55.707937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:55.707937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:55.707937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:33:55.707937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:33:55.707937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:33:55.707937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:33:55.707937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:33:55.707937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:33:55.707937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:33:55.707937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:33:55.707937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 582/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:33:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:55.711415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e71fa-1b71-4ed1-a042-068fba706df0 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:55.811516 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:55.837172 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e71fa-1b71-4ed1-a042-068fba706df0 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:55.877191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4244998-35fd-4fdb-aeb8-8c81a5642ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:55.877805 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4244998-35fd-4fdb-aeb8-8c81a5642ade None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:55.878079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4244998-35fd-4fdb-aeb8-8c81a5642ade None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:55.878346 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4244998-35fd-4fdb-aeb8-8c81a5642ade None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:55.879041 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4244998-35fd-4fdb-aeb8-8c81a5642ade None admin] RBAC: Adding query filter params (name=admin) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:55.879338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4244998-35fd-4fdb-aeb8-8c81a5642ade None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:55.880719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4244998-35fd-4fdb-aeb8-8c81a5642ade None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:55.890806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 575/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:33:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:55.897992 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73009a94-3ab4-4d39-b018-22b2ca6c5128 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:56.009330 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73009a94-3ab4-4d39-b018-22b2ca6c5128 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:56.044469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807decf-f626-4e52-acde-fce0f6f7ed19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:56.045105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807decf-f626-4e52-acde-fce0f6f7ed19 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:56.045382 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807decf-f626-4e52-acde-fce0f6f7ed19 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:56.045653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807decf-f626-4e52-acde-fce0f6f7ed19 None admin] PATH_INFO: `/v3/users/085c647729604e268464241be65a66f0` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:56.046812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5807decf-f626-4e52-acde-fce0f6f7ed19 None admin] RBAC: Authorizing `identity:get_user(user_id=085c647729604e268464241be65a66f0)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:56.048257 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5807decf-f626-4e52-acde-fce0f6f7ed19 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:56.049881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 583/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:33:55 2022] GET /identity/v3/users/085c647729604e268464241be65a66f0 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:56.056630 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4244998-35fd-4fdb-aeb8-8c81a5642ade None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:56.166429 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4244998-35fd-4fdb-aeb8-8c81a5642ade None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:56.201982 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98608005-5a40-4f9b-83fb-d96fad15976f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:56.202642 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98608005-5a40-4f9b-83fb-d96fad15976f None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:56.202922 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98608005-5a40-4f9b-83fb-d96fad15976f None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:56.203191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98608005-5a40-4f9b-83fb-d96fad15976f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:56.216258 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:56.286815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98608005-5a40-4f9b-83fb-d96fad15976f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:56.290359 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98608005-5a40-4f9b-83fb-d96fad15976f None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:56.458373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 576/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:33:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 16:33:56.465553 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807decf-f626-4e52-acde-fce0f6f7ed19 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:56.494389 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:56.596567 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807decf-f626-4e52-acde-fce0f6f7ed19 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:56.634554 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d8879-b949-429f-b3f4-92aa1eb753bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:56.635235 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d8879-b949-429f-b3f4-92aa1eb753bb None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:56.635514 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d8879-b949-429f-b3f4-92aa1eb753bb None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:56.635785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d8879-b949-429f-b3f4-92aa1eb753bb None admin] PATH_INFO: `/v3/projects/1d0eb32ba0494f7c914511315fdaa594` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:56.637018 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d8879-b949-429f-b3f4-92aa1eb753bb None admin] RBAC: Authorizing `identity:get_project(project_id=1d0eb32ba0494f7c914511315fdaa594)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:56.638458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d8879-b949-429f-b3f4-92aa1eb753bb None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:56.640143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 584/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:33:56 2022] GET /identity/v3/projects/1d0eb32ba0494f7c914511315fdaa594 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:56.646419 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98608005-5a40-4f9b-83fb-d96fad15976f None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:56.692993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:56.753316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98608005-5a40-4f9b-83fb-d96fad15976f None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:56.792130 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654c90b-ba84-4457-84da-d06c64610a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:56.792729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654c90b-ba84-4457-84da-d06c64610a6e None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:56.793007 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654c90b-ba84-4457-84da-d06c64610a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:56.793274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654c90b-ba84-4457-84da-d06c64610a6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:56.794002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654c90b-ba84-4457-84da-d06c64610a6e None admin] RBAC: Adding query filter params (role.id=e578ef831127463d9b48cc89fe20329c, scope.project.id=1d0eb32ba0494f7c914511315fdaa594, user.id=085c647729604e268464241be65a66f0) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:56.794305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654c90b-ba84-4457-84da-d06c64610a6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:56.795782 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654c90b-ba84-4457-84da-d06c64610a6e None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:56.801921 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 577/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:33:56 2022] GET /identity/v3/role_assignments?user.id=085c647729604e268464241be65a66f0&scope.project.id=1d0eb32ba0494f7c914511315fdaa594&role.id=e578ef831127463d9b48cc89fe20329c => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:33:58.909636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c427b4-fc07-4e24-94b8-335d2d6c0bbd None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:58.910275 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c427b4-fc07-4e24-94b8-335d2d6c0bbd None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:58.910275 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c427b4-fc07-4e24-94b8-335d2d6c0bbd None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:58.911797 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 585/1162] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:33:58.917707 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf7e27-2a5f-4bbd-ace1-c1533c9e51a3 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:58.918285 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf7e27-2a5f-4bbd-ace1-c1533c9e51a3 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:58.918285 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf7e27-2a5f-4bbd-ace1-c1533c9e51a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:58.944014 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-47cf7e27-2a5f-4bbd-ace1-c1533c9e51a3 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:33:58.946613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-47cf7e27-2a5f-4bbd-ace1-c1533c9e51a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:33:59.057575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 578/1163] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:33:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:33:59.079991 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c427b4-fc07-4e24-94b8-335d2d6c0bbd None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:59.173258 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:59.193814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c427b4-fc07-4e24-94b8-335d2d6c0bbd None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:59.239833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4e3be-b31e-4c23-b6a5-cc2afce0015b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:59.241422 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4e3be-b31e-4c23-b6a5-cc2afce0015b None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:59.241918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4e3be-b31e-4c23-b6a5-cc2afce0015b None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:59.242393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4e3be-b31e-4c23-b6a5-cc2afce0015b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:59.316284 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4e3be-b31e-4c23-b6a5-cc2afce0015b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:59.317740 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4e3be-b31e-4c23-b6a5-cc2afce0015b None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:59.404899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:59.424351 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 586/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:33:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:33:59.431587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf7e27-2a5f-4bbd-ace1-c1533c9e51a3 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:59.466364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:33:59.538297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf7e27-2a5f-4bbd-ace1-c1533c9e51a3 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:59.579604 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5795e-0ad4-457f-9f2f-f21ff4651fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:59.580229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5795e-0ad4-457f-9f2f-f21ff4651fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:59.580505 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5795e-0ad4-457f-9f2f-f21ff4651fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:59.580775 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5795e-0ad4-457f-9f2f-f21ff4651fd8 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:59.588616 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f5795e-0ad4-457f-9f2f-f21ff4651fd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:59.590075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f5795e-0ad4-457f-9f2f-f21ff4651fd8 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:59.602983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-a4f5795e-0ad4-457f-9f2f-f21ff4651fd8 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 08 16:33:59.602983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:59.602983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 08 16:33:59.602983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 08 16:33:59.602983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:33:59.602983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:33:59.602983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:33:59.602983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:33:59.602983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 08 16:33:59.602983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:59.602983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:33:59.602983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:59.602983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:33:59.602983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:33:59.602983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:33:59.602983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:33:59.602983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:33:59.604246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:33:59.604246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:33:59.604246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:33:59.604246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:33:59.604246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:33:59.604246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:33:59.604246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:33:59.604246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:33:59.604246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:33:59.604246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:33:59.604246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:33:59.604246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:33:59.604246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:33:59.604246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:33:59.604246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:33:59.604246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:33:59.605178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:33:59.605178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:33:59.605178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:33:59.605178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:33:59.605178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:33:59.605178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:33:59.605178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:33:59.605178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:33:59.605178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:33:59.605178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:33:59.605178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 08 16:33:59.605178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:33:59.605178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 08 16:33:59.605178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:33:59.605178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 579/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Aug 8 16:33:59 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:33:59.610561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4e3be-b31e-4c23-b6a5-cc2afce0015b None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:33:59.727156 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4e3be-b31e-4c23-b6a5-cc2afce0015b None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:33:59.766852 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d326677-20b5-4a74-9005-d8a20b69b003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:33:59.767526 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d326677-20b5-4a74-9005-d8a20b69b003 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:33:59.767844 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d326677-20b5-4a74-9005-d8a20b69b003 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:33:59.768155 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d326677-20b5-4a74-9005-d8a20b69b003 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:33:59.768857 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d326677-20b5-4a74-9005-d8a20b69b003 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:33:59.769194 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d326677-20b5-4a74-9005-d8a20b69b003 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:33:59.770757 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d326677-20b5-4a74-9005-d8a20b69b003 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:33:59.779817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 587/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Aug 8 16:33:59 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:34:01.858542 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0cb0f-e536-46bf-8e2c-fd37f0a5458c None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:01.859160 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0cb0f-e536-46bf-8e2c-fd37f0a5458c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:01.859160 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0cb0f-e536-46bf-8e2c-fd37f0a5458c None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:01.860725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 580/1167] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:34:01.867218 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e932a-b268-4cf4-bac8-76ddd3816af1 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:01.867868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e932a-b268-4cf4-bac8-76ddd3816af1 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:01.867868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e932a-b268-4cf4-bac8-76ddd3816af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:01.887965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-5c6e932a-b268-4cf4-bac8-76ddd3816af1 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:34:01.889298 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-5c6e932a-b268-4cf4-bac8-76ddd3816af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:34:01.937103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:34:01.964011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 588/1168] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:34:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:34:01.990147 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0cb0f-e536-46bf-8e2c-fd37f0a5458c None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:34:02.092770 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:34:02.101171 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0cb0f-e536-46bf-8e2c-fd37f0a5458c None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:34:02.139355 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790dffb-5fb2-406e-9417-6b234bd3d7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:34:02.139957 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790dffb-5fb2-406e-9417-6b234bd3d7cb None admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:02.140230 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790dffb-5fb2-406e-9417-6b234bd3d7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:02.140493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790dffb-5fb2-406e-9417-6b234bd3d7cb None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:02.141151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b790dffb-5fb2-406e-9417-6b234bd3d7cb None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:34:02.142522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b790dffb-5fb2-406e-9417-6b234bd3d7cb None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:34:02.248268 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 581/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Mon Aug 8 16:34:01 2022] POST /identity/v3/domains => generated 267 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:34:04.362050 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b502723-3652-4bd9-b292-5519d8753bb0 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:04.362616 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b502723-3652-4bd9-b292-5519d8753bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:04.362831 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b502723-3652-4bd9-b292-5519d8753bb0 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:04.364274 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 589/1170] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:34:04.370680 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed4586-6510-4454-947e-41e9ae3f1e3d None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:04.371186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed4586-6510-4454-947e-41e9ae3f1e3d None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:04.371306 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed4586-6510-4454-947e-41e9ae3f1e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:04.391274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-ebed4586-6510-4454-947e-41e9ae3f1e3d None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:34:04.392479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ebed4586-6510-4454-947e-41e9ae3f1e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:34:04.464617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 582/1171] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:34:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:34:04.489273 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b502723-3652-4bd9-b292-5519d8753bb0 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:34:04.597182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b502723-3652-4bd9-b292-5519d8753bb0 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:34:04.634867 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35707911-87ab-4c17-8338-f7bca3fd59ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:34:04.635404 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35707911-87ab-4c17-8338-f7bca3fd59ab None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:04.635680 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35707911-87ab-4c17-8338-f7bca3fd59ab None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:04.635951 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35707911-87ab-4c17-8338-f7bca3fd59ab None admin] PATH_INFO: `/v3/domains/8dc7c25deacc474cb0b81aecd7d6f4ff` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:04.647999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35707911-87ab-4c17-8338-f7bca3fd59ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=8dc7c25deacc474cb0b81aecd7d6f4ff)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:34:04.649470 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35707911-87ab-4c17-8338-f7bca3fd59ab None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:34:04.651234 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 590/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Mon Aug 8 16:34:04 2022] GET /identity/v3/domains/8dc7c25deacc474cb0b81aecd7d6f4ff => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:34:04.657582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed4586-6510-4454-947e-41e9ae3f1e3d None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:34:04.749445 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:34:04.766467 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed4586-6510-4454-947e-41e9ae3f1e3d None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:34:04.804047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad84c5-6868-45fc-a1d1-e64a55aa86b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:34:04.804722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad84c5-6868-45fc-a1d1-e64a55aa86b6 None admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:04.805071 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad84c5-6868-45fc-a1d1-e64a55aa86b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:04.805355 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad84c5-6868-45fc-a1d1-e64a55aa86b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:04.806063 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad84c5-6868-45fc-a1d1-e64a55aa86b6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:34:04.807822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad84c5-6868-45fc-a1d1-e64a55aa86b6 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:34:04.902283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 583/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Aug 8 16:34:04 2022] POST /identity/v3/projects => generated 361 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:34:07.005633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd904d-1439-49d8-a95e-eff6f4b3c189 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:07.006787 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd904d-1439-49d8-a95e-eff6f4b3c189 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:07.006787 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd904d-1439-49d8-a95e-eff6f4b3c189 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:07.008189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 591/1174] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:34:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:34:07.015013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aec2e3-779e-4dd4-8614-68c26484aab7 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:07.015527 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aec2e3-779e-4dd4-8614-68c26484aab7 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:07.015982 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aec2e3-779e-4dd4-8614-68c26484aab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:07.045468 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-84aec2e3-779e-4dd4-8614-68c26484aab7 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:34:07.047512 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-84aec2e3-779e-4dd4-8614-68c26484aab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:34:07.162350 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 584/1175] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:34:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:34:07.188187 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd904d-1439-49d8-a95e-eff6f4b3c189 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:34:07.232176 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:34:07.308421 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd904d-1439-49d8-a95e-eff6f4b3c189 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:34:07.347085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5100c-cdee-4456-8548-9f96f2b8a78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:34:07.347793 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5100c-cdee-4456-8548-9f96f2b8a78d None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:07.348112 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5100c-cdee-4456-8548-9f96f2b8a78d None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:07.348424 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5100c-cdee-4456-8548-9f96f2b8a78d None admin] PATH_INFO: `/v3/domains/8dc7c25deacc474cb0b81aecd7d6f4ff` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:07.349692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5100c-cdee-4456-8548-9f96f2b8a78d None admin] RBAC: Authorizing `identity:get_domain(domain_id=8dc7c25deacc474cb0b81aecd7d6f4ff)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:34:07.351911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5100c-cdee-4456-8548-9f96f2b8a78d None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:34:07.353664 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 592/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Mon Aug 8 16:34:07 2022] GET /identity/v3/domains/8dc7c25deacc474cb0b81aecd7d6f4ff => generated 267 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:34:07.359997 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aec2e3-779e-4dd4-8614-68c26484aab7 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:34:07.432044 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:34:07.470091 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aec2e3-779e-4dd4-8614-68c26484aab7 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:34:07.507731 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a8676-d0b3-4a2a-b193-d689e1381c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:34:07.508341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a8676-d0b3-4a2a-b193-d689e1381c0b None admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:07.508613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a8676-d0b3-4a2a-b193-d689e1381c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:07.508874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a8676-d0b3-4a2a-b193-d689e1381c0b None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:07.509566 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a8676-d0b3-4a2a-b193-d689e1381c0b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:34:07.511052 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a8676-d0b3-4a2a-b193-d689e1381c0b None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:34:07.626834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 585/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Aug 8 16:34:07 2022] POST /identity/v3/users => generated 311 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:34:09.789280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8256d-05af-4c81-81c2-abe047a2cfba None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:09.789852 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8256d-05af-4c81-81c2-abe047a2cfba None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:09.789852 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8256d-05af-4c81-81c2-abe047a2cfba None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:09.791195 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 593/1178] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:34:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:34:09.796857 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0b4fe-1dcb-4223-9593-1bfe40d45eba None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:09.797336 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0b4fe-1dcb-4223-9593-1bfe40d45eba None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:09.797479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0b4fe-1dcb-4223-9593-1bfe40d45eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:09.816512 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-8fd0b4fe-1dcb-4223-9593-1bfe40d45eba None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:34:09.817790 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-8fd0b4fe-1dcb-4223-9593-1bfe40d45eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:34:09.892289 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 586/1179] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:34:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:34:09.917200 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8256d-05af-4c81-81c2-abe047a2cfba None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:34:10.029769 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8256d-05af-4c81-81c2-abe047a2cfba None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:34:10.032302 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:34:10.067601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4eb92-a9b3-4d58-87a0-73c6da458d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:34:10.068208 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4eb92-a9b3-4d58-87a0-73c6da458d2b None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:10.068491 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4eb92-a9b3-4d58-87a0-73c6da458d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:10.069220 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4eb92-a9b3-4d58-87a0-73c6da458d2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:10.080756 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4eb92-a9b3-4d58-87a0-73c6da458d2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:34:10.086231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-68e4eb92-a9b3-4d58-87a0-73c6da458d2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:34:10.086231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:34:10.086231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:34:10.086231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:34:10.086231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:34:10.086231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:34:10.086231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:34:10.086231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:34:10.086231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:34:10.086231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:34:10.086231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:34:10.086231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:34:10.086231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:34:10.086231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:34:10.086231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:34:10.086231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:34:10.087851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:34:10.087851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:34:10.087851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:34:10.087851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:34:10.087851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:34:10.087851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:34:10.087851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:34:10.087851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:34:10.087851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:34:10.087851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:34:10.087851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:34:10.087851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:34:10.087851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:34:10.087851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:34:10.087851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:34:10.087851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:34:10.089674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:34:10.089674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:34:10.089674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:34:10.089674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:34:10.089674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:34:10.089674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:34:10.089674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:34:10.089674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:34:10.089674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:34:10.089674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:34:10.089674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:34:10.089674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:34:10.089674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 594/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:34:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:34:10.095639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0b4fe-1dcb-4223-9593-1bfe40d45eba None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:34:10.153059 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:34:10.202331 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0b4fe-1dcb-4223-9593-1bfe40d45eba None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:34:10.240190 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0eeaa-95ad-402b-bb33-df9e7aad6b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:34:10.241327 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0eeaa-95ad-402b-bb33-df9e7aad6b93 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:10.241651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0eeaa-95ad-402b-bb33-df9e7aad6b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:10.241961 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0eeaa-95ad-402b-bb33-df9e7aad6b93 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:10.242701 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0eeaa-95ad-402b-bb33-df9e7aad6b93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:34:10.243040 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0eeaa-95ad-402b-bb33-df9e7aad6b93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:34:10.244496 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0eeaa-95ad-402b-bb33-df9e7aad6b93 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:34:10.253122 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 587/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:34:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:34:10.262438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4eb92-a9b3-4d58-87a0-73c6da458d2b None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:34:10.373435 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4eb92-a9b3-4d58-87a0-73c6da458d2b None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:34:10.409708 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5a26e-fbea-402f-a953-9251f0746f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:34:10.410413 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5a26e-fbea-402f-a953-9251f0746f6a None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:10.410763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5a26e-fbea-402f-a953-9251f0746f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:10.411075 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5a26e-fbea-402f-a953-9251f0746f6a None admin] PATH_INFO: `/v3/users/6a2e55b27d7748558dc2259d5d9ec9fd` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:10.435350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5a26e-fbea-402f-a953-9251f0746f6a None admin] RBAC: Authorizing `identity:get_user(user_id=6a2e55b27d7748558dc2259d5d9ec9fd)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:34:10.438065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5a26e-fbea-402f-a953-9251f0746f6a None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:34:10.441240 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 595/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:34:10 2022] GET /identity/v3/users/6a2e55b27d7748558dc2259d5d9ec9fd => generated 311 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:34:10.448342 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0eeaa-95ad-402b-bb33-df9e7aad6b93 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:34:10.580341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0eeaa-95ad-402b-bb33-df9e7aad6b93 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:34:10.595791 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:34:10.617229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8033dc9-4cc8-4ee8-a487-606f39939dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:34:10.617891 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8033dc9-4cc8-4ee8-a487-606f39939dcd None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:10.618215 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8033dc9-4cc8-4ee8-a487-606f39939dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:10.618488 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8033dc9-4cc8-4ee8-a487-606f39939dcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:10.690537 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8033dc9-4cc8-4ee8-a487-606f39939dcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:34:10.691976 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8033dc9-4cc8-4ee8-a487-606f39939dcd None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:34:10.800541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 588/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:34:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 16:34:10.811112 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5a26e-fbea-402f-a953-9251f0746f6a None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:34:10.854680 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:34:10.937950 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5a26e-fbea-402f-a953-9251f0746f6a None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:34:10.977363 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d80c65-feab-4c1d-9564-fe793a7cbc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:34:10.977920 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d80c65-feab-4c1d-9564-fe793a7cbc00 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:10.978196 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d80c65-feab-4c1d-9564-fe793a7cbc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:10.978465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d80c65-feab-4c1d-9564-fe793a7cbc00 None admin] PATH_INFO: `/v3/projects/6b211ab2504642d1944ac6bf5bbb1a1e` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:10.979774 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d80c65-feab-4c1d-9564-fe793a7cbc00 None admin] RBAC: Authorizing `identity:get_project(project_id=6b211ab2504642d1944ac6bf5bbb1a1e)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:34:10.981116 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d80c65-feab-4c1d-9564-fe793a7cbc00 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:34:10.983925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 596/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:34:10 2022] GET /identity/v3/projects/6b211ab2504642d1944ac6bf5bbb1a1e => generated 361 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:34:10.989488 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8033dc9-4cc8-4ee8-a487-606f39939dcd None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:34:11.007914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:34:11.097556 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8033dc9-4cc8-4ee8-a487-606f39939dcd None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:34:11.134977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb952e7-b378-41a8-bc4a-a3fa51e845af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:34:11.135937 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb952e7-b378-41a8-bc4a-a3fa51e845af None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:11.136222 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb952e7-b378-41a8-bc4a-a3fa51e845af None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:11.136494 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb952e7-b378-41a8-bc4a-a3fa51e845af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:11.137230 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb952e7-b378-41a8-bc4a-a3fa51e845af None admin] RBAC: Adding query filter params (role.id=e578ef831127463d9b48cc89fe20329c, scope.project.id=6b211ab2504642d1944ac6bf5bbb1a1e, user.id=6a2e55b27d7748558dc2259d5d9ec9fd) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:34:11.137537 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb952e7-b378-41a8-bc4a-a3fa51e845af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:34:11.139022 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb952e7-b378-41a8-bc4a-a3fa51e845af None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:34:11.145306 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 589/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:34:10 2022] GET /identity/v3/role_assignments?user.id=6a2e55b27d7748558dc2259d5d9ec9fd&scope.project.id=6b211ab2504642d1944ac6bf5bbb1a1e&role.id=e578ef831127463d9b48cc89fe20329c => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:34:13.211027 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98ec2f-e739-44ab-a1a5-1c6ec01f6bee None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:13.211625 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98ec2f-e739-44ab-a1a5-1c6ec01f6bee None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:13.211625 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98ec2f-e739-44ab-a1a5-1c6ec01f6bee None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:13.212806 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 597/1186] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:34:13.218910 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2d9c0-26b5-42c1-a857-f442dfa10c48 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:13.219416 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2d9c0-26b5-42c1-a857-f442dfa10c48 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:13.219545 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2d9c0-26b5-42c1-a857-f442dfa10c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:13.238752 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-e1a2d9c0-26b5-42c1-a857-f442dfa10c48 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:34:13.240022 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-e1a2d9c0-26b5-42c1-a857-f442dfa10c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:34:13.313614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 590/1187] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:34:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:34:13.337160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98ec2f-e739-44ab-a1a5-1c6ec01f6bee None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:34:13.439239 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:34:13.450835 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98ec2f-e739-44ab-a1a5-1c6ec01f6bee None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:34:13.489432 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7a6be-4cb6-4b76-984c-126485576f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:34:13.490326 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7a6be-4cb6-4b76-984c-126485576f1c None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:13.491294 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7a6be-4cb6-4b76-984c-126485576f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:13.491728 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7a6be-4cb6-4b76-984c-126485576f1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:13.502790 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c7a6be-4cb6-4b76-984c-126485576f1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:34:13.508147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-07c7a6be-4cb6-4b76-984c-126485576f1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:34:13.508147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:34:13.508147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:34:13.508147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:34:13.508147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:34:13.508147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:34:13.508147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:34:13.508147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:34:13.508147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:34:13.508147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:34:13.508147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:34:13.508147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:34:13.508147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:34:13.508147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:34:13.508147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:34:13.508147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:34:13.509958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:34:13.509958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:34:13.509958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:34:13.509958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:34:13.509958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:34:13.509958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:34:13.509958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:34:13.509958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:34:13.509958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:34:13.509958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:34:13.509958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:34:13.509958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:34:13.509958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:34:13.509958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:34:13.509958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:34:13.509958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:34:13.511203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:34:13.511203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:34:13.511203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:34:13.511203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:34:13.511203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:34:13.511203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:34:13.511203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:34:13.511203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:34:13.511203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:34:13.511203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:34:13.511203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:34:13.511203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:34:13.511203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 598/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:34:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:34:13.515698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2d9c0-26b5-42c1-a857-f442dfa10c48 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:34:13.526642 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:34:13.634870 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2d9c0-26b5-42c1-a857-f442dfa10c48 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:34:13.694844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db91fc4-7274-43cd-9bc8-266c69bc216a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:34:13.695457 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db91fc4-7274-43cd-9bc8-266c69bc216a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:13.695733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db91fc4-7274-43cd-9bc8-266c69bc216a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:13.695998 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db91fc4-7274-43cd-9bc8-266c69bc216a None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:13.696665 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db91fc4-7274-43cd-9bc8-266c69bc216a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:34:13.696963 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db91fc4-7274-43cd-9bc8-266c69bc216a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:34:13.698341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db91fc4-7274-43cd-9bc8-266c69bc216a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:34:13.707213 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 591/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:34:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:34:13.713587 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7a6be-4cb6-4b76-984c-126485576f1c None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:34:13.817663 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7a6be-4cb6-4b76-984c-126485576f1c None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:34:13.852386 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b5d1c-842f-4395-9c36-c153d65543ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:34:13.853017 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b5d1c-842f-4395-9c36-c153d65543ac None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:13.853288 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b5d1c-842f-4395-9c36-c153d65543ac None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:13.853554 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b5d1c-842f-4395-9c36-c153d65543ac None admin] PATH_INFO: `/v3/users/6a2e55b27d7748558dc2259d5d9ec9fd` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:13.854704 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b5d1c-842f-4395-9c36-c153d65543ac None admin] RBAC: Authorizing `identity:get_user(user_id=6a2e55b27d7748558dc2259d5d9ec9fd)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:34:13.856132 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b5d1c-842f-4395-9c36-c153d65543ac None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:34:13.857709 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 599/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:34:13 2022] GET /identity/v3/users/6a2e55b27d7748558dc2259d5d9ec9fd => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:34:13.863777 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db91fc4-7274-43cd-9bc8-266c69bc216a None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:34:13.948254 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:34:13.991685 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db91fc4-7274-43cd-9bc8-266c69bc216a None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:34:14.035837 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888752b2-6332-47c2-9a0d-8e258507517a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:34:14.036481 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888752b2-6332-47c2-9a0d-8e258507517a None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:14.036752 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888752b2-6332-47c2-9a0d-8e258507517a None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:14.037013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888752b2-6332-47c2-9a0d-8e258507517a None admin] PATH_INFO: `/v3/projects/6b211ab2504642d1944ac6bf5bbb1a1e` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:14.039642 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888752b2-6332-47c2-9a0d-8e258507517a None admin] RBAC: Authorizing `identity:get_project(project_id=6b211ab2504642d1944ac6bf5bbb1a1e)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:34:14.041205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888752b2-6332-47c2-9a0d-8e258507517a None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:34:14.043117 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 592/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:34:13 2022] GET /identity/v3/projects/6b211ab2504642d1944ac6bf5bbb1a1e => generated 361 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:34:14.049558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b5d1c-842f-4395-9c36-c153d65543ac None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:34:14.083847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:34:14.202932 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b5d1c-842f-4395-9c36-c153d65543ac None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:34:14.239744 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d1ff1-642b-47be-84b4-3eeed6c587f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:34:14.240920 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d1ff1-642b-47be-84b4-3eeed6c587f4 None admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:14.241206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d1ff1-642b-47be-84b4-3eeed6c587f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:14.241206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d1ff1-642b-47be-84b4-3eeed6c587f4 None admin] PATH_INFO: `/v3/projects/6b211ab2504642d1944ac6bf5bbb1a1e/users/6a2e55b27d7748558dc2259d5d9ec9fd/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:14.243603 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d1ff1-642b-47be-84b4-3eeed6c587f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=6b211ab2504642d1944ac6bf5bbb1a1e, user_id=6a2e55b27d7748558dc2259d5d9ec9fd, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:34:14.245049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d1ff1-642b-47be-84b4-3eeed6c587f4 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:34:14.335728 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 600/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:34:14 2022] PUT /identity/v3/projects/6b211ab2504642d1944ac6bf5bbb1a1e/users/6a2e55b27d7748558dc2259d5d9ec9fd/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:34:16.405554 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b19c7-fe6e-4df3-88af-e66c43807fe2 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:16.406159 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b19c7-fe6e-4df3-88af-e66c43807fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:16.406478 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b19c7-fe6e-4df3-88af-e66c43807fe2 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:16.409131 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 593/1193] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:34:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:34:16.415391 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe25619-7ceb-4cea-a316-51a6fff17907 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:16.416015 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe25619-7ceb-4cea-a316-51a6fff17907 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:16.416015 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe25619-7ceb-4cea-a316-51a6fff17907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:16.435528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-ebe25619-7ceb-4cea-a316-51a6fff17907 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:34:16.436751 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ebe25619-7ceb-4cea-a316-51a6fff17907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:34:16.511843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 601/1194] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:34:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:34:16.536341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b19c7-fe6e-4df3-88af-e66c43807fe2 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:34:16.662062 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b19c7-fe6e-4df3-88af-e66c43807fe2 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:34:16.698642 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfb3c8-4fe2-475d-bd75-05757c51ff9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:34:16.699269 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfb3c8-4fe2-475d-bd75-05757c51ff9f None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:16.699539 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfb3c8-4fe2-475d-bd75-05757c51ff9f None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:16.699800 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfb3c8-4fe2-475d-bd75-05757c51ff9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:16.707204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:34:16.711566 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bfb3c8-4fe2-475d-bd75-05757c51ff9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:34:16.716394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-d1bfb3c8-4fe2-475d-bd75-05757c51ff9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:34:16.716394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:34:16.716394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:34:16.716394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:34:16.716394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:34:16.716394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:34:16.716394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:34:16.716394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:34:16.716394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:34:16.716394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:34:16.716394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:34:16.716394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:34:16.716394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:34:16.716394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:34:16.716394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:34:16.716394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:34:16.724564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:34:16.724564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:34:16.724564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:34:16.724564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:34:16.724564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:34:16.724564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:34:16.724564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:34:16.724564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:34:16.724564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:34:16.724564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:34:16.724564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:34:16.724564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:34:16.724564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:34:16.724564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:34:16.724564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:34:16.724564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:34:16.726553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:34:16.726553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:34:16.726553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:34:16.726553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:34:16.726553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:34:16.726553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:34:16.726553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:34:16.726553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:34:16.726553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:34:16.726553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:34:16.726553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:34:16.726553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:34:16.726553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 594/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:34:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:34:16.727374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe25619-7ceb-4cea-a316-51a6fff17907 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:34:16.735550 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:34:16.838115 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe25619-7ceb-4cea-a316-51a6fff17907 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:34:16.873057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b5149-95a8-4d32-9795-503ba5a143c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:34:16.873670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b5149-95a8-4d32-9795-503ba5a143c1 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:16.873945 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b5149-95a8-4d32-9795-503ba5a143c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:16.874270 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b5149-95a8-4d32-9795-503ba5a143c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:16.874907 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b5149-95a8-4d32-9795-503ba5a143c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:34:16.875207 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b5149-95a8-4d32-9795-503ba5a143c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:34:16.876571 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b5149-95a8-4d32-9795-503ba5a143c1 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:34:16.885893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 602/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:34:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:34:16.892091 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfb3c8-4fe2-475d-bd75-05757c51ff9f None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:34:17.001157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfb3c8-4fe2-475d-bd75-05757c51ff9f None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:34:17.036535 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5fffc3-820d-4442-a695-f798ad84e62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:34:17.037165 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5fffc3-820d-4442-a695-f798ad84e62e None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:17.037502 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5fffc3-820d-4442-a695-f798ad84e62e None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:17.037770 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5fffc3-820d-4442-a695-f798ad84e62e None admin] PATH_INFO: `/v3/users/6a2e55b27d7748558dc2259d5d9ec9fd` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:17.038999 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5fffc3-820d-4442-a695-f798ad84e62e None admin] RBAC: Authorizing `identity:get_user(user_id=6a2e55b27d7748558dc2259d5d9ec9fd)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:34:17.040434 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5fffc3-820d-4442-a695-f798ad84e62e None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:34:17.042055 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 595/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Aug 8 16:34:16 2022] GET /identity/v3/users/6a2e55b27d7748558dc2259d5d9ec9fd => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:34:17.048195 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b5149-95a8-4d32-9795-503ba5a143c1 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:34:17.121576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:34:17.152882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b5149-95a8-4d32-9795-503ba5a143c1 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:34:17.188933 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a751fc-4a95-4422-9d2b-dd92d68ffd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:34:17.189450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a751fc-4a95-4422-9d2b-dd92d68ffd0b None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:17.189733 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a751fc-4a95-4422-9d2b-dd92d68ffd0b None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:17.189992 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a751fc-4a95-4422-9d2b-dd92d68ffd0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:17.263385 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a751fc-4a95-4422-9d2b-dd92d68ffd0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:34:17.264822 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a751fc-4a95-4422-9d2b-dd92d68ffd0b None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:34:17.331135 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:34:17.370956 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 603/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:34:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:34:17.377381 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5fffc3-820d-4442-a695-f798ad84e62e None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:34:17.467697 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:34:17.512294 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5fffc3-820d-4442-a695-f798ad84e62e None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:34:17.554939 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b77e2-18ba-4b42-87f0-cc92895eb985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:34:17.555583 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b77e2-18ba-4b42-87f0-cc92895eb985 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:17.555856 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b77e2-18ba-4b42-87f0-cc92895eb985 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:17.556121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b77e2-18ba-4b42-87f0-cc92895eb985 None admin] PATH_INFO: `/v3/projects/6b211ab2504642d1944ac6bf5bbb1a1e` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:17.557371 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b77e2-18ba-4b42-87f0-cc92895eb985 None admin] RBAC: Authorizing `identity:get_project(project_id=6b211ab2504642d1944ac6bf5bbb1a1e)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:34:17.558844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b77e2-18ba-4b42-87f0-cc92895eb985 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:34:17.560517 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 596/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Aug 8 16:34:17 2022] GET /identity/v3/projects/6b211ab2504642d1944ac6bf5bbb1a1e => generated 361 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:34:17.567124 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a751fc-4a95-4422-9d2b-dd92d68ffd0b None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:34:17.674247 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a751fc-4a95-4422-9d2b-dd92d68ffd0b None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:34:17.712679 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e255304-ad5d-44fb-baac-7aa47d78f9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:34:17.713306 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e255304-ad5d-44fb-baac-7aa47d78f9aa None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:34:17.713582 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e255304-ad5d-44fb-baac-7aa47d78f9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:34:17.713848 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e255304-ad5d-44fb-baac-7aa47d78f9aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:34:17.714605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e255304-ad5d-44fb-baac-7aa47d78f9aa None admin] RBAC: Adding query filter params (role.id=e578ef831127463d9b48cc89fe20329c, scope.project.id=6b211ab2504642d1944ac6bf5bbb1a1e, user.id=6a2e55b27d7748558dc2259d5d9ec9fd) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:34:17.714912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e255304-ad5d-44fb-baac-7aa47d78f9aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:34:17.716355 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e255304-ad5d-44fb-baac-7aa47d78f9aa None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:34:17.723448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 604/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:34:17 2022] GET /identity/v3/role_assignments?user.id=6a2e55b27d7748558dc2259d5d9ec9fd&scope.project.id=6b211ab2504642d1944ac6bf5bbb1a1e&role.id=e578ef831127463d9b48cc89fe20329c => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:34:27.549045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=33 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:34:27.720125 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=83 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:32.210798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62106f74-4dda-4c39-a5a9-ddb507ca07c6 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:32.211384 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62106f74-4dda-4c39-a5a9-ddb507ca07c6 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:32.211384 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62106f74-4dda-4c39-a5a9-ddb507ca07c6 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:32.212630 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 597/1201] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:35:32.218140 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b92286-b70e-4822-8ef8-4b6df12c9c02 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:32.218649 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b92286-b70e-4822-8ef8-4b6df12c9c02 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:32.218765 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b92286-b70e-4822-8ef8-4b6df12c9c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:32.237775 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-59b92286-b70e-4822-8ef8-4b6df12c9c02 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:35:32.239019 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-59b92286-b70e-4822-8ef8-4b6df12c9c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:35:32.309967 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 605/1202] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:35:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:35:32.335490 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62106f74-4dda-4c39-a5a9-ddb507ca07c6 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:32.448075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62106f74-4dda-4c39-a5a9-ddb507ca07c6 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:32.483965 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089b888-c0fe-4266-baa7-c97a3a07e9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:32.484504 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089b888-c0fe-4266-baa7-c97a3a07e9c8 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:32.484778 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089b888-c0fe-4266-baa7-c97a3a07e9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:32.485041 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089b888-c0fe-4266-baa7-c97a3a07e9c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:32.496096 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089b888-c0fe-4266-baa7-c97a3a07e9c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:32.497574 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089b888-c0fe-4266-baa7-c97a3a07e9c8 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:32.513824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-5089b888-c0fe-4266-baa7-c97a3a07e9c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:32.513824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:32.513824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:32.513824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:32.513824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:32.513824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:32.513824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:32.513824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:32.513824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:32.513824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:32.513824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:32.513824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:32.513824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:35:32.513824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:35:32.513824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:35:32.513824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:35:32.515237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:32.515237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:32.515237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:32.515237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:32.515237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:32.515237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:32.515237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:32.515237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:32.515237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:32.515237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:32.515237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:32.515237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:32.515237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:32.515237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:32.515237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:32.515237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:32.515237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:35:32.516536 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:35:32.516536 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:32.516536 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:35:32.516536 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 598/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:32.522135 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b92286-b70e-4822-8ef8-4b6df12c9c02 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:32.622321 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b92286-b70e-4822-8ef8-4b6df12c9c02 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:32.654372 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:32.660282 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e71c6-e50e-4335-a7a5-35372890ddf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:32.661010 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e71c6-e50e-4335-a7a5-35372890ddf9 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:32.661286 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e71c6-e50e-4335-a7a5-35372890ddf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:32.661556 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e71c6-e50e-4335-a7a5-35372890ddf9 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:32.662201 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e71c6-e50e-4335-a7a5-35372890ddf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:32.662494 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e71c6-e50e-4335-a7a5-35372890ddf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:32.663885 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e71c6-e50e-4335-a7a5-35372890ddf9 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:32.675785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 606/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:32.686930 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089b888-c0fe-4266-baa7-c97a3a07e9c8 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:32.743452 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:32.794769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089b888-c0fe-4266-baa7-c97a3a07e9c8 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:32.830958 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19594892-fb8b-4d1d-9538-10e46218beae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:32.831565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19594892-fb8b-4d1d-9538-10e46218beae None admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:32.831890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19594892-fb8b-4d1d-9538-10e46218beae None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:32.832155 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19594892-fb8b-4d1d-9538-10e46218beae None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:32.832814 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19594892-fb8b-4d1d-9538-10e46218beae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:32.834480 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19594892-fb8b-4d1d-9538-10e46218beae None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:33.074492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 599/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Aug 8 16:35:32 2022] POST /identity/v3/users => generated 251 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:35:35.178249 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e74758d-c4b1-4d12-99d9-f685be762371 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:35.178874 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e74758d-c4b1-4d12-99d9-f685be762371 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:35.178874 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e74758d-c4b1-4d12-99d9-f685be762371 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:35.180172 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 607/1206] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:35:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:35:35.185939 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24428cef-bf79-464e-95e7-391d5e340048 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:35.185939 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24428cef-bf79-464e-95e7-391d5e340048 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:35.186443 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24428cef-bf79-464e-95e7-391d5e340048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:35.204232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-24428cef-bf79-464e-95e7-391d5e340048 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:35:35.205457 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-24428cef-bf79-464e-95e7-391d5e340048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:35:35.275041 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 600/1207] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:35:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:35:35.299495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e74758d-c4b1-4d12-99d9-f685be762371 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:35.415209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e74758d-c4b1-4d12-99d9-f685be762371 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:35.462021 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4537f-6e3d-4d4f-82f1-f9b96208771c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:35.462652 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4537f-6e3d-4d4f-82f1-f9b96208771c None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:35.462931 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4537f-6e3d-4d4f-82f1-f9b96208771c None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:35.463200 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4537f-6e3d-4d4f-82f1-f9b96208771c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:35.474871 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4537f-6e3d-4d4f-82f1-f9b96208771c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:35:35.479690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-38b4537f-6e3d-4d4f-82f1-f9b96208771c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:35:35.479690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:35.479690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:35.479690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:35.479690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:35.479690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:35.479690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:35.479690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:35.479690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:35.479690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:35.479690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:35.479690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:35.479690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:35:35.479690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:35:35.479690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:35:35.479690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:35:35.480714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:35:35.480714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:35:35.480714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:35:35.480714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:35:35.480714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:35.480714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:35.480714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:35.480714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:35.480714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:35.480714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:35.480714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:35.480714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:35.480714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:35.480714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:35.480714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:35.480714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:35.481777 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:35.481777 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:35.481777 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:35.481777 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:35.481777 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:35:35.481777 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:35:35.481777 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:35:35.481777 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:35:35.481777 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:35:35.481777 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:35:35.481777 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:35:35.481777 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:35:35.481777 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 608/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:35:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:35.487104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24428cef-bf79-464e-95e7-391d5e340048 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:35.532153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:35.604315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24428cef-bf79-464e-95e7-391d5e340048 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:35.646703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299555a9-3deb-404c-8326-a0ca29930207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:35.647307 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299555a9-3deb-404c-8326-a0ca29930207 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:35.647586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299555a9-3deb-404c-8326-a0ca29930207 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:35.647856 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299555a9-3deb-404c-8326-a0ca29930207 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:35.648512 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299555a9-3deb-404c-8326-a0ca29930207 None admin] RBAC: Adding query filter params (name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:35.648812 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299555a9-3deb-404c-8326-a0ca29930207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:35.650222 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299555a9-3deb-404c-8326-a0ca29930207 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:35.663625 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 601/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Aug 8 16:35:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:35.669529 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4537f-6e3d-4d4f-82f1-f9b96208771c None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:35.721892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:35.775783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4537f-6e3d-4d4f-82f1-f9b96208771c None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:35.811932 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82579dce-45a9-46b7-9460-79b28d45a8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:35.812711 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82579dce-45a9-46b7-9460-79b28d45a8cf None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:35.812992 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82579dce-45a9-46b7-9460-79b28d45a8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:35.813261 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82579dce-45a9-46b7-9460-79b28d45a8cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:35.824637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82579dce-45a9-46b7-9460-79b28d45a8cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:35.826151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82579dce-45a9-46b7-9460-79b28d45a8cf None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:35.846902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-82579dce-45a9-46b7-9460-79b28d45a8cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:35.846902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:35.846902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:35.846902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:35.846902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:35.846902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:35.846902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:35.846902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:35.846902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:35.846902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:35.846902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:35.846902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:35.846902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:35:35.846902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:35:35.846902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:35:35.846902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:35:35.848072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:35.848072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:35.848072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:35.848072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:35.848072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:35.848072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:35.848072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:35.848072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:35.848072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:35.848072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:35.848072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:35.848072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:35.848072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:35.848072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:35.848072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:35.848072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:35.848072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:35:35.850439 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:35:35.850439 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:35.850439 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:35:35.850439 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 609/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:35:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:35.855503 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299555a9-3deb-404c-8326-a0ca29930207 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:35.956450 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:35.960438 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299555a9-3deb-404c-8326-a0ca29930207 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:35.994994 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541df11c-b167-41ac-a72c-098f8db03490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:35.995521 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541df11c-b167-41ac-a72c-098f8db03490 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:35.995801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541df11c-b167-41ac-a72c-098f8db03490 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:35.996065 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541df11c-b167-41ac-a72c-098f8db03490 None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:35.996880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541df11c-b167-41ac-a72c-098f8db03490 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:35.997010 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541df11c-b167-41ac-a72c-098f8db03490 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:35.998469 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541df11c-b167-41ac-a72c-098f8db03490 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:36.010479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 602/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:36.016565 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82579dce-45a9-46b7-9460-79b28d45a8cf None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:36.118053 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82579dce-45a9-46b7-9460-79b28d45a8cf None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:36.119679 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:36.153245 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0940e1f-4079-4af7-b386-0ba99b6b80c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:36.153975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0940e1f-4079-4af7-b386-0ba99b6b80c5 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:36.154253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0940e1f-4079-4af7-b386-0ba99b6b80c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:36.154522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0940e1f-4079-4af7-b386-0ba99b6b80c5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:36.162296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0940e1f-4079-4af7-b386-0ba99b6b80c5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:36.163770 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0940e1f-4079-4af7-b386-0ba99b6b80c5 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:36.174921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-b0940e1f-4079-4af7-b386-0ba99b6b80c5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:35:36.174921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:36.174921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:36.174921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:36.174921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:36.174921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:36.174921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:36.174921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:36.174921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:36.174921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:36.174921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:36.174921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:36.174921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:35:36.174921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:35:36.174921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:35:36.174921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:35:36.176117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:36.176117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:36.176117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:35:36.176117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:35:36.176117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:35:36.176117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:35:36.176117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:36.176117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:36.176117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:36.176117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:36.176117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:36.176117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:36.176117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:36.176117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:36.176117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:36.176117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:36.176117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:36.176117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:36.177415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:36.177415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:36.177415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:35:36.177415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:35:36.177415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:35:36.177415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:35:36.177415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:35:36.177415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:35:36.177415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:35:36.177415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:35:36.177415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 610/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:35:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:36.182044 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541df11c-b167-41ac-a72c-098f8db03490 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:36.285184 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541df11c-b167-41ac-a72c-098f8db03490 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:36.319265 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f876340-b5aa-4d2d-8a57-bd3cc7d1aacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:36.319865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f876340-b5aa-4d2d-8a57-bd3cc7d1aacd None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:36.320139 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f876340-b5aa-4d2d-8a57-bd3cc7d1aacd None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:36.320405 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f876340-b5aa-4d2d-8a57-bd3cc7d1aacd None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:36.321119 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f876340-b5aa-4d2d-8a57-bd3cc7d1aacd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:36.321416 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f876340-b5aa-4d2d-8a57-bd3cc7d1aacd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:36.322885 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f876340-b5aa-4d2d-8a57-bd3cc7d1aacd None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:36.344278 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 603/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Aug 8 16:35:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:36.350128 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0940e1f-4079-4af7-b386-0ba99b6b80c5 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:36.452634 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0940e1f-4079-4af7-b386-0ba99b6b80c5 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:36.487057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d9725-8d9c-4fdd-b661-b0df64fe4896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:36.487687 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d9725-8d9c-4fdd-b661-b0df64fe4896 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:36.487965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d9725-8d9c-4fdd-b661-b0df64fe4896 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:36.488250 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d9725-8d9c-4fdd-b661-b0df64fe4896 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:36.522666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:36.562461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d9725-8d9c-4fdd-b661-b0df64fe4896 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:36.563884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d9725-8d9c-4fdd-b661-b0df64fe4896 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:36.672803 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 611/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:35:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:35:36.679497 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f876340-b5aa-4d2d-8a57-bd3cc7d1aacd None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:36.700457 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:36.785432 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f876340-b5aa-4d2d-8a57-bd3cc7d1aacd None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:36.821828 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d939afe-e9c7-4279-960d-62e47383392f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:36.822377 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d939afe-e9c7-4279-960d-62e47383392f None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:36.822681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d939afe-e9c7-4279-960d-62e47383392f None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:36.822946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d939afe-e9c7-4279-960d-62e47383392f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:36.833213 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d939afe-e9c7-4279-960d-62e47383392f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:36.834694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d939afe-e9c7-4279-960d-62e47383392f None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:36.850780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-3d939afe-e9c7-4279-960d-62e47383392f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:36.850780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:36.850780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:36.850780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:36.850780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:36.850780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:36.850780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:36.850780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:36.850780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:36.850780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:36.850780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:36.850780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:36.850780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:35:36.850780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:35:36.850780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:35:36.850780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:35:36.852083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:36.852083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:36.852083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:36.852083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:36.852083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:36.852083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:36.852083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:36.852083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:36.852083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:36.852083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:36.852083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:36.852083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:36.852083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:36.852083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:36.852083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:36.852083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:36.852083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:35:36.853194 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:35:36.853194 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:36.853194 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:35:36.853194 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 604/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:36.857654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d9725-8d9c-4fdd-b661-b0df64fe4896 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:36.919438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:36.962254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d9725-8d9c-4fdd-b661-b0df64fe4896 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:36.995856 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001436d3-b2d0-4bb0-b2c8-5a6bf663d326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:36.996450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001436d3-b2d0-4bb0-b2c8-5a6bf663d326 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:36.996727 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001436d3-b2d0-4bb0-b2c8-5a6bf663d326 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:36.996997 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001436d3-b2d0-4bb0-b2c8-5a6bf663d326 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:36.997627 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001436d3-b2d0-4bb0-b2c8-5a6bf663d326 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:36.997920 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001436d3-b2d0-4bb0-b2c8-5a6bf663d326 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:36.999308 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001436d3-b2d0-4bb0-b2c8-5a6bf663d326 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:37.012643 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 612/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:37.018233 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d939afe-e9c7-4279-960d-62e47383392f None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:37.090262 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:37.121690 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d939afe-e9c7-4279-960d-62e47383392f None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:37.156428 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d75be0-2d45-4aba-9f23-85e530e53708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:37.157049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d75be0-2d45-4aba-9f23-85e530e53708 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:37.157416 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d75be0-2d45-4aba-9f23-85e530e53708 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:37.157699 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d75be0-2d45-4aba-9f23-85e530e53708 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:37.164649 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d75be0-2d45-4aba-9f23-85e530e53708 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:37.166181 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d75be0-2d45-4aba-9f23-85e530e53708 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:37.178720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-44d75be0-2d45-4aba-9f23-85e530e53708 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:35:37.178720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:37.178720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:37.178720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:37.178720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:37.178720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:37.178720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:37.178720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:37.178720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:37.178720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:37.178720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:37.178720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:37.178720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:35:37.178720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:35:37.178720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:35:37.179722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:35:37.179722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:37.179722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:37.179722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:37.179722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:37.179722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:37.179722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:37.179722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:37.179722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:37.179722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:37.179722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:37.179722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:37.179722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:37.179722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:37.179722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:37.179722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:37.179722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:37.180704 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:35:37.180704 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:35:37.180704 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:35:37.180704 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:35:37.180704 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:35:37.180704 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:35:37.180704 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:35:37.180704 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:35:37.181456 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 605/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:35:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:37.186185 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001436d3-b2d0-4bb0-b2c8-5a6bf663d326 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:37.290458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001436d3-b2d0-4bb0-b2c8-5a6bf663d326 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:37.313885 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:37.327378 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ccc1b-18e5-45b0-bada-fe9b9cf0e53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:37.328122 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ccc1b-18e5-45b0-bada-fe9b9cf0e53c None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:37.328395 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ccc1b-18e5-45b0-bada-fe9b9cf0e53c None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:37.328803 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ccc1b-18e5-45b0-bada-fe9b9cf0e53c None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:37.329718 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ccc1b-18e5-45b0-bada-fe9b9cf0e53c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:37.330021 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ccc1b-18e5-45b0-bada-fe9b9cf0e53c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:37.331536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ccc1b-18e5-45b0-bada-fe9b9cf0e53c None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:37.344245 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 613/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:35:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:35:37.350479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d75be0-2d45-4aba-9f23-85e530e53708 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:37.454682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d75be0-2d45-4aba-9f23-85e530e53708 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:37.491502 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:37.497419 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e522b3f-cb5a-4160-9d34-98df8a27b989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:37.498021 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e522b3f-cb5a-4160-9d34-98df8a27b989 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:37.498294 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e522b3f-cb5a-4160-9d34-98df8a27b989 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:37.498557 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e522b3f-cb5a-4160-9d34-98df8a27b989 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:37.499307 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e522b3f-cb5a-4160-9d34-98df8a27b989 None admin] RBAC: Adding query filter params (role.id=7764573472d34ecc8bedf7a9c7935170, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=6e74dbb121d749749cf5f9d9d7221008) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:37.499609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e522b3f-cb5a-4160-9d34-98df8a27b989 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:37.501113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e522b3f-cb5a-4160-9d34-98df8a27b989 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:37.507238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 606/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:35:37 2022] GET /identity/v3/role_assignments?user.id=6e74dbb121d749749cf5f9d9d7221008&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=7764573472d34ecc8bedf7a9c7935170 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:39.493806 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c971976-e06a-435a-98f7-a5df109fbf41 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:39.494351 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c971976-e06a-435a-98f7-a5df109fbf41 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:39.494351 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c971976-e06a-435a-98f7-a5df109fbf41 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:39.495703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 614/1220] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:35:39.501073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a74e7-1be1-4d99-947e-4cbe0c28ea71 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:39.502700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a74e7-1be1-4d99-947e-4cbe0c28ea71 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:39.502700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a74e7-1be1-4d99-947e-4cbe0c28ea71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:39.521413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-9b9a74e7-1be1-4d99-947e-4cbe0c28ea71 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:35:39.522657 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-9b9a74e7-1be1-4d99-947e-4cbe0c28ea71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:35:39.591238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 607/1221] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:35:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:35:39.615030 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c971976-e06a-435a-98f7-a5df109fbf41 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:39.727673 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c971976-e06a-435a-98f7-a5df109fbf41 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:39.768756 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98fe8d-77ac-4b37-b6fa-97b0cd2d1d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:39.769323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98fe8d-77ac-4b37-b6fa-97b0cd2d1d46 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:39.769599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98fe8d-77ac-4b37-b6fa-97b0cd2d1d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:39.769868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98fe8d-77ac-4b37-b6fa-97b0cd2d1d46 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:39.782296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f98fe8d-77ac-4b37-b6fa-97b0cd2d1d46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:35:39.787312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-8f98fe8d-77ac-4b37-b6fa-97b0cd2d1d46 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:35:39.787312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:39.787312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:39.787312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:39.787312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:39.787312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:39.787312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:39.787312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:39.787312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:39.787312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:39.787312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:39.787312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:39.787312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:35:39.787312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:35:39.787312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:35:39.787312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:35:39.788472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:35:39.788472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:35:39.788472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:35:39.788472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:35:39.788472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:39.788472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:39.788472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:39.788472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:39.788472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:39.788472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:39.788472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:39.788472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:39.788472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:39.788472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:39.788472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:39.788472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:39.807149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:39.807149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:39.807149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:39.807149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:39.807149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:35:39.807149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:35:39.807149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:35:39.807149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:35:39.807149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:35:39.807149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:35:39.807149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:35:39.807149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:35:39.807149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 615/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:35:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:39.807957 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a74e7-1be1-4d99-947e-4cbe0c28ea71 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:39.908667 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a74e7-1be1-4d99-947e-4cbe0c28ea71 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:39.939264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:39.950398 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89728a-850e-403d-bd2f-6722c97abe9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:39.951043 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89728a-850e-403d-bd2f-6722c97abe9b None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:39.951317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89728a-850e-403d-bd2f-6722c97abe9b None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:39.951581 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89728a-850e-403d-bd2f-6722c97abe9b None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:39.952246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89728a-850e-403d-bd2f-6722c97abe9b None admin] RBAC: Adding query filter params (name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:39.952542 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89728a-850e-403d-bd2f-6722c97abe9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:39.953966 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89728a-850e-403d-bd2f-6722c97abe9b None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:39.963084 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 608/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Aug 8 16:35:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:39.969746 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98fe8d-77ac-4b37-b6fa-97b0cd2d1d46 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:40.018268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:40.090225 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98fe8d-77ac-4b37-b6fa-97b0cd2d1d46 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:40.146102 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fe102b-ff34-42b3-87ed-1075224f9ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:40.147066 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe102b-ff34-42b3-87ed-1075224f9ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:40.147488 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe102b-ff34-42b3-87ed-1075224f9ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:40.147870 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe102b-ff34-42b3-87ed-1075224f9ca4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:40.162826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fe102b-ff34-42b3-87ed-1075224f9ca4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:40.166910 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fe102b-ff34-42b3-87ed-1075224f9ca4 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:40.190485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-48fe102b-ff34-42b3-87ed-1075224f9ca4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:40.190485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:40.190485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:40.190485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:40.190485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:40.190485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:40.190485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:40.190485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:40.190485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:40.190485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:40.190485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:40.190485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:40.190485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:35:40.190485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:35:40.190485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:35:40.190485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:35:40.191645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:40.191645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:40.191645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:40.191645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:40.191645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:40.191645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:40.191645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:40.191645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:40.191645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:40.191645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:40.191645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:40.191645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:40.191645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:40.191645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:40.191645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:40.191645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:40.191645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:35:40.192680 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:35:40.192680 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:40.192680 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:35:40.193060 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 616/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:40.198992 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89728a-850e-403d-bd2f-6722c97abe9b None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:40.302907 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89728a-850e-403d-bd2f-6722c97abe9b None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:40.335744 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdcc10d-2031-4e2d-868f-bd10ed590eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:40.336335 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcc10d-2031-4e2d-868f-bd10ed590eaf None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:40.336605 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcc10d-2031-4e2d-868f-bd10ed590eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:40.336867 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcc10d-2031-4e2d-868f-bd10ed590eaf None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:40.337481 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdcc10d-2031-4e2d-868f-bd10ed590eaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:40.337771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdcc10d-2031-4e2d-868f-bd10ed590eaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:40.339160 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdcc10d-2031-4e2d-868f-bd10ed590eaf None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:40.356071 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 609/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:40.357939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fe102b-ff34-42b3-87ed-1075224f9ca4 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:40.453296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:40.460636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fe102b-ff34-42b3-87ed-1075224f9ca4 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:40.495222 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e46193-4248-418d-a439-7cbcb393cb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:40.495955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e46193-4248-418d-a439-7cbcb393cb8d None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:40.496169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e46193-4248-418d-a439-7cbcb393cb8d None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:40.496478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e46193-4248-418d-a439-7cbcb393cb8d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:40.504173 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e46193-4248-418d-a439-7cbcb393cb8d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:40.506111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e46193-4248-418d-a439-7cbcb393cb8d None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:40.517451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-53e46193-4248-418d-a439-7cbcb393cb8d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:35:40.517451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:40.517451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:40.517451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:40.517451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:40.517451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:40.517451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:40.517451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:40.517451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:40.517451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:40.517451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:40.517451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:40.517451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:35:40.517451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:35:40.517451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:35:40.517451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:35:40.520471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:40.520471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:40.520471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:35:40.520471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:35:40.520471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:35:40.520471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:35:40.520471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:40.520471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:40.520471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:40.520471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:40.520471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:40.520471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:40.520471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:40.520471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:40.520471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:40.520471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:40.520471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:40.520471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:40.522136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:40.522136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:40.522136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:35:40.522136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:35:40.522136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:35:40.522136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:35:40.522136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:35:40.522136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:35:40.522136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:35:40.522136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:35:40.522136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 617/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:35:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:35:40.525338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdcc10d-2031-4e2d-868f-bd10ed590eaf None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:40.573131 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:40.625219 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdcc10d-2031-4e2d-868f-bd10ed590eaf None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:40.659771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfc680-02e0-4623-b294-e273a71aaeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:40.660378 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfc680-02e0-4623-b294-e273a71aaeb8 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:40.660815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfc680-02e0-4623-b294-e273a71aaeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:40.661087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfc680-02e0-4623-b294-e273a71aaeb8 None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:40.661904 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfc680-02e0-4623-b294-e273a71aaeb8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:40.662233 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfc680-02e0-4623-b294-e273a71aaeb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:40.663700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfc680-02e0-4623-b294-e273a71aaeb8 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:40.681521 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 610/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Aug 8 16:35:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:40.687400 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e46193-4248-418d-a439-7cbcb393cb8d None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:40.795220 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e46193-4248-418d-a439-7cbcb393cb8d None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:40.829730 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd338f8d-43bd-4962-8de8-b2e8f5ebd166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:40.830426 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd338f8d-43bd-4962-8de8-b2e8f5ebd166 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:40.830729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd338f8d-43bd-4962-8de8-b2e8f5ebd166 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:40.830998 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd338f8d-43bd-4962-8de8-b2e8f5ebd166 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:40.842630 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd338f8d-43bd-4962-8de8-b2e8f5ebd166 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:40.844070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd338f8d-43bd-4962-8de8-b2e8f5ebd166 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:40.858447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-fd338f8d-43bd-4962-8de8-b2e8f5ebd166 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:40.858447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:40.858447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:40.858447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:40.858447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:40.858447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:40.858447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:40.858447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:40.858447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:40.858447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:40.858447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:40.858447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:40.858447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:35:40.858447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:35:40.858447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:35:40.858447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:35:40.859498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:40.859498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:40.859498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:40.859498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:40.859498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:40.859498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:40.859498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:40.859498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:40.859498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:40.859498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:40.859498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:40.859498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:40.859498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:40.859498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:40.859498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:40.859498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:40.859498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:35:40.860590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:35:40.860590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:40.860590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:35:40.860590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 618/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:35:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:40.865703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfc680-02e0-4623-b294-e273a71aaeb8 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:40.970975 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfc680-02e0-4623-b294-e273a71aaeb8 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:40.973400 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:41.006324 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4841348-33e5-48ba-8629-cf509f75f139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:41.006956 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4841348-33e5-48ba-8629-cf509f75f139 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:41.007227 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4841348-33e5-48ba-8629-cf509f75f139 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:41.007489 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4841348-33e5-48ba-8629-cf509f75f139 None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:41.008113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4841348-33e5-48ba-8629-cf509f75f139 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:41.008406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4841348-33e5-48ba-8629-cf509f75f139 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:41.009770 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4841348-33e5-48ba-8629-cf509f75f139 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:41.021347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 611/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:41.027405 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd338f8d-43bd-4962-8de8-b2e8f5ebd166 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:41.037159 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:41.133783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd338f8d-43bd-4962-8de8-b2e8f5ebd166 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:41.166764 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2966371-8db3-49f8-b811-fe1fe06d3f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:41.167389 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2966371-8db3-49f8-b811-fe1fe06d3f25 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:41.167661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2966371-8db3-49f8-b811-fe1fe06d3f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:41.167923 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2966371-8db3-49f8-b811-fe1fe06d3f25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:41.174375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2966371-8db3-49f8-b811-fe1fe06d3f25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:41.175855 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2966371-8db3-49f8-b811-fe1fe06d3f25 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:41.187925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-c2966371-8db3-49f8-b811-fe1fe06d3f25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:35:41.187925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:41.187925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:41.187925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:41.187925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:41.187925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:41.187925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:41.187925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:41.187925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:41.187925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:41.187925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:41.187925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:41.187925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:35:41.187925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:35:41.187925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:35:41.188960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:35:41.188960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:41.188960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:41.188960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:41.188960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:41.188960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:41.188960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:41.188960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:41.188960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:41.188960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:41.188960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:41.188960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:41.188960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:41.188960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:41.188960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:41.188960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:41.188960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:41.190227 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:35:41.190227 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:35:41.190227 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:35:41.190227 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:35:41.190227 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:35:41.190227 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:35:41.190227 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:35:41.190227 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:35:41.190227 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 619/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:35:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:41.195829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4841348-33e5-48ba-8629-cf509f75f139 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:41.296179 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4841348-33e5-48ba-8629-cf509f75f139 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:41.329578 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34349ac1-f997-476b-855f-8cd2af739a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:41.330172 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34349ac1-f997-476b-855f-8cd2af739a49 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:41.330443 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34349ac1-f997-476b-855f-8cd2af739a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:41.330742 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34349ac1-f997-476b-855f-8cd2af739a49 None admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:41.331418 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34349ac1-f997-476b-855f-8cd2af739a49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:41.331767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34349ac1-f997-476b-855f-8cd2af739a49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:41.333831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34349ac1-f997-476b-855f-8cd2af739a49 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:41.346450 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 612/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:35:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:35:41.353265 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2966371-8db3-49f8-b811-fe1fe06d3f25 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:41.422302 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:41.459811 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2966371-8db3-49f8-b811-fe1fe06d3f25 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:41.495078 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb8a19-26fb-480a-ab5a-d98b16652e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:41.495733 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb8a19-26fb-480a-ab5a-d98b16652e4f None admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:41.496296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb8a19-26fb-480a-ab5a-d98b16652e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:41.496296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb8a19-26fb-480a-ab5a-d98b16652e4f None admin] PATH_INFO: `/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/6e74dbb121d749749cf5f9d9d7221008/roles/7764573472d34ecc8bedf7a9c7935170` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:41.510823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb8a19-26fb-480a-ab5a-d98b16652e4f None admin] RBAC: Authorizing `identity:create_grant(project_id=56ab2fc79d964f4d86edefb4d1b2c993, user_id=6e74dbb121d749749cf5f9d9d7221008, role_id=7764573472d34ecc8bedf7a9c7935170)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:41.512379 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb8a19-26fb-480a-ab5a-d98b16652e4f None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:41.662849 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 620/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:35:41 2022] PUT /identity/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/6e74dbb121d749749cf5f9d9d7221008/roles/7764573472d34ecc8bedf7a9c7935170 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:35:43.688647 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadf487-528d-460a-ab0e-b4f413aa1146 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:43.689191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadf487-528d-460a-ab0e-b4f413aa1146 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:43.689191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadf487-528d-460a-ab0e-b4f413aa1146 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:43.690552 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 613/1233] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:35:43.696366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107ad2a-db83-4a6f-ba0f-262166440f8b None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:43.696811 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107ad2a-db83-4a6f-ba0f-262166440f8b None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:43.696922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107ad2a-db83-4a6f-ba0f-262166440f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:43.715652 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-5107ad2a-db83-4a6f-ba0f-262166440f8b None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:35:43.717109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-5107ad2a-db83-4a6f-ba0f-262166440f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:35:43.787251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 621/1234] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:35:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:35:43.810205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadf487-528d-460a-ab0e-b4f413aa1146 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:43.837922 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:43.921848 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadf487-528d-460a-ab0e-b4f413aa1146 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:43.960282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1179d66e-4937-4e6c-92a7-df7710334538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:43.960835 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179d66e-4937-4e6c-92a7-df7710334538 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:43.961107 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179d66e-4937-4e6c-92a7-df7710334538 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:43.961370 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179d66e-4937-4e6c-92a7-df7710334538 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:43.972910 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1179d66e-4937-4e6c-92a7-df7710334538 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:35:43.977988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-1179d66e-4937-4e6c-92a7-df7710334538 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:35:43.977988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:43.977988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:43.977988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:43.977988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:43.977988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:43.977988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:43.977988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:43.977988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:43.977988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:43.977988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:43.977988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:43.977988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:35:43.977988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:35:43.977988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:35:43.977988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:35:43.979277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:35:43.979277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:35:43.979277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:35:43.979277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:35:43.979277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:43.979277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:43.979277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:43.979277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:43.979277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:43.979277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:43.979277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:43.979277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:43.979277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:43.979277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:43.979277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:43.979277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:43.980298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:43.980298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:43.980298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:43.980298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:43.980298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:35:43.980298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:35:43.980298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:35:43.980298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:35:43.980298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:35:43.980298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:35:43.980298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:35:43.980298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:35:43.980298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 614/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Aug 8 16:35:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:43.985484 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107ad2a-db83-4a6f-ba0f-262166440f8b None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:44.032556 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:44.083453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107ad2a-db83-4a6f-ba0f-262166440f8b None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:44.117302 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df84ebe-ed56-48d2-ad93-5fb8fab6c073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:44.117910 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df84ebe-ed56-48d2-ad93-5fb8fab6c073 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:44.118181 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df84ebe-ed56-48d2-ad93-5fb8fab6c073 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:44.118448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df84ebe-ed56-48d2-ad93-5fb8fab6c073 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:44.119133 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df84ebe-ed56-48d2-ad93-5fb8fab6c073 None admin] RBAC: Adding query filter params (name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:44.119429 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df84ebe-ed56-48d2-ad93-5fb8fab6c073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:44.120792 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df84ebe-ed56-48d2-ad93-5fb8fab6c073 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:44.128660 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 622/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Aug 8 16:35:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:44.135340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1179d66e-4937-4e6c-92a7-df7710334538 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:44.224041 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:44.239711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1179d66e-4937-4e6c-92a7-df7710334538 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:44.273589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d3565-510a-4c25-81c4-b62ce7e2b6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:44.274214 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d3565-510a-4c25-81c4-b62ce7e2b6b1 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:44.274486 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d3565-510a-4c25-81c4-b62ce7e2b6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:44.274778 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d3565-510a-4c25-81c4-b62ce7e2b6b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:44.284664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d3565-510a-4c25-81c4-b62ce7e2b6b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:44.286316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d3565-510a-4c25-81c4-b62ce7e2b6b1 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:44.300564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-d40d3565-510a-4c25-81c4-b62ce7e2b6b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:44.300564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:44.300564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:44.300564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:44.300564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:44.300564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:44.300564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:44.300564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:44.300564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:44.300564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:44.300564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:44.300564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:44.300564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:35:44.300564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:35:44.300564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:35:44.300564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:35:44.302983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:44.302983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:44.302983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:44.302983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:44.302983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:44.302983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:44.302983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:44.302983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:44.302983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:44.302983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:44.302983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:44.302983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:44.302983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:44.302983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:44.302983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:44.302983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:44.302983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:35:44.303915 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:35:44.303915 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:44.303915 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:35:44.303915 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 615/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:44.308192 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df84ebe-ed56-48d2-ad93-5fb8fab6c073 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:44.408706 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df84ebe-ed56-48d2-ad93-5fb8fab6c073 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:44.417827 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:44.442531 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d714c-780b-425f-b274-09c32c0dcab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:44.443164 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d714c-780b-425f-b274-09c32c0dcab9 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:44.443436 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d714c-780b-425f-b274-09c32c0dcab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:44.443702 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d714c-780b-425f-b274-09c32c0dcab9 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:44.444330 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d714c-780b-425f-b274-09c32c0dcab9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:44.444631 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d714c-780b-425f-b274-09c32c0dcab9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:44.445987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d714c-780b-425f-b274-09c32c0dcab9 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:44.458103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 623/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:44.464132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d3565-510a-4c25-81c4-b62ce7e2b6b1 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:44.564727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d3565-510a-4c25-81c4-b62ce7e2b6b1 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:44.600129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d656c-6193-4ef0-a14e-89862184e18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:44.600778 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d656c-6193-4ef0-a14e-89862184e18c None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:44.601045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d656c-6193-4ef0-a14e-89862184e18c None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:44.601303 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d656c-6193-4ef0-a14e-89862184e18c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:44.604079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:44.609178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d656c-6193-4ef0-a14e-89862184e18c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:44.610668 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d656c-6193-4ef0-a14e-89862184e18c None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:44.623681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-c73d656c-6193-4ef0-a14e-89862184e18c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:35:44.623681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:44.623681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:44.623681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:44.623681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:44.623681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:44.623681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:44.623681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:44.623681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:44.623681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:44.623681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:44.623681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:44.623681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:35:44.623681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:35:44.623681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:35:44.623681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:35:44.636235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:44.636235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:44.636235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:35:44.636235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:35:44.636235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:35:44.636235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:35:44.636235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:44.636235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:44.636235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:44.636235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:44.636235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:44.636235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:44.636235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:44.636235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:44.636235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:44.636235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:44.636235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:44.636235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:44.637475 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:44.637475 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:44.637475 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:35:44.637475 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:35:44.637475 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:35:44.637475 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:35:44.637475 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:35:44.637475 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:35:44.637475 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:35:44.637475 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:35:44.637475 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 616/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:35:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:44.638235 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d714c-780b-425f-b274-09c32c0dcab9 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:44.736654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d714c-780b-425f-b274-09c32c0dcab9 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:44.773451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f4592-c47b-43fc-a2d7-c6738d203fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:44.774055 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f4592-c47b-43fc-a2d7-c6738d203fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:44.774331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f4592-c47b-43fc-a2d7-c6738d203fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:44.774623 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f4592-c47b-43fc-a2d7-c6738d203fa6 None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:44.775347 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f4592-c47b-43fc-a2d7-c6738d203fa6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:44.775647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f4592-c47b-43fc-a2d7-c6738d203fa6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:44.777073 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f4592-c47b-43fc-a2d7-c6738d203fa6 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:44.795131 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 624/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Aug 8 16:35:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:44.802143 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d656c-6193-4ef0-a14e-89862184e18c None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:44.904093 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d656c-6193-4ef0-a14e-89862184e18c None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:44.941774 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bd6d3-c68d-45b1-ad68-ba2c32657537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:44.943237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bd6d3-c68d-45b1-ad68-ba2c32657537 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:44.943519 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bd6d3-c68d-45b1-ad68-ba2c32657537 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:44.943788 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bd6d3-c68d-45b1-ad68-ba2c32657537 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:45.005860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:45.012524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122bd6d3-c68d-45b1-ad68-ba2c32657537 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:45.013993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122bd6d3-c68d-45b1-ad68-ba2c32657537 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:45.117744 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 617/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:35:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 16:35:45.123632 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f4592-c47b-43fc-a2d7-c6738d203fa6 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:45.151961 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:45.224029 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f4592-c47b-43fc-a2d7-c6738d203fa6 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:45.257267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492d031-7cab-4ef5-8f3f-85b5e5fd562d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:45.257913 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492d031-7cab-4ef5-8f3f-85b5e5fd562d None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:45.258182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492d031-7cab-4ef5-8f3f-85b5e5fd562d None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:45.258447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492d031-7cab-4ef5-8f3f-85b5e5fd562d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:45.268100 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492d031-7cab-4ef5-8f3f-85b5e5fd562d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:45.269540 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492d031-7cab-4ef5-8f3f-85b5e5fd562d None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:45.284007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-d492d031-7cab-4ef5-8f3f-85b5e5fd562d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:45.284007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:45.284007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:45.284007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:45.284007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:45.284007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:45.284007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:45.284007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:45.284007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:45.284007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:45.284007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:45.284007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:45.284007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:35:45.284007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:35:45.284007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:35:45.284007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:35:45.286217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:45.286217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:45.286217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:45.286217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:45.286217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:45.286217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:45.286217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:45.286217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:45.286217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:45.286217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:45.286217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:45.286217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:45.286217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:45.286217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:45.286217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:45.286217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:45.286217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:35:45.287652 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:35:45.287652 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:45.287652 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:35:45.287652 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 625/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:35:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:45.291615 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bd6d3-c68d-45b1-ad68-ba2c32657537 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:45.386800 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:45.392842 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bd6d3-c68d-45b1-ad68-ba2c32657537 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:45.429127 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6ba4f-a5a1-4abe-b7da-37b44a3332aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:45.429728 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6ba4f-a5a1-4abe-b7da-37b44a3332aa None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:45.429999 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6ba4f-a5a1-4abe-b7da-37b44a3332aa None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:45.430263 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6ba4f-a5a1-4abe-b7da-37b44a3332aa None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:45.430925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce6ba4f-a5a1-4abe-b7da-37b44a3332aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:45.431218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce6ba4f-a5a1-4abe-b7da-37b44a3332aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:45.432583 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce6ba4f-a5a1-4abe-b7da-37b44a3332aa None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:45.443695 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 618/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:45.449653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492d031-7cab-4ef5-8f3f-85b5e5fd562d None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:45.533903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:45.557677 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492d031-7cab-4ef5-8f3f-85b5e5fd562d None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:45.595766 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3355a-5a4c-4a8a-9003-ee63405f74da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:45.596426 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3355a-5a4c-4a8a-9003-ee63405f74da None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:45.596704 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3355a-5a4c-4a8a-9003-ee63405f74da None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:45.596987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3355a-5a4c-4a8a-9003-ee63405f74da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:45.604674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3355a-5a4c-4a8a-9003-ee63405f74da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:45.606131 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3355a-5a4c-4a8a-9003-ee63405f74da None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:45.618538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-2aa3355a-5a4c-4a8a-9003-ee63405f74da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:35:45.618538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:45.618538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:45.618538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:45.618538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:45.618538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:45.618538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:45.618538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:45.618538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:45.618538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:45.618538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:45.618538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:45.618538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:35:45.618538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:35:45.618538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:35:45.619732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:35:45.619732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:45.619732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:45.619732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:45.619732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:45.619732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:45.619732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:45.619732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:45.619732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:45.619732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:45.619732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:45.619732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:45.619732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:45.619732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:45.619732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:45.619732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:45.619732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:45.621158 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:35:45.621158 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:35:45.621158 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:35:45.621158 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:35:45.621158 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:35:45.621158 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:35:45.621158 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:35:45.621158 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:35:45.621158 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 626/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:35:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:45.626835 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6ba4f-a5a1-4abe-b7da-37b44a3332aa None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:45.729632 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6ba4f-a5a1-4abe-b7da-37b44a3332aa None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:45.764560 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13874452-2888-4735-a298-08bcb9e3034e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:45.765160 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13874452-2888-4735-a298-08bcb9e3034e None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:45.765464 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13874452-2888-4735-a298-08bcb9e3034e None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:45.765729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13874452-2888-4735-a298-08bcb9e3034e None admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:45.766407 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13874452-2888-4735-a298-08bcb9e3034e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:45.766741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13874452-2888-4735-a298-08bcb9e3034e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:45.768193 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13874452-2888-4735-a298-08bcb9e3034e None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:45.780215 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 619/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:35:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:45.786434 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3355a-5a4c-4a8a-9003-ee63405f74da None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:45.887427 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3355a-5a4c-4a8a-9003-ee63405f74da None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:45.921085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652dcb09-b110-4346-bed0-97c7f21d16ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:45.921707 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652dcb09-b110-4346-bed0-97c7f21d16ba None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:45.924971 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652dcb09-b110-4346-bed0-97c7f21d16ba None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:45.924971 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652dcb09-b110-4346-bed0-97c7f21d16ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:45.924971 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652dcb09-b110-4346-bed0-97c7f21d16ba None admin] RBAC: Adding query filter params (role.id=7764573472d34ecc8bedf7a9c7935170, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=6e74dbb121d749749cf5f9d9d7221008) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:45.924971 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652dcb09-b110-4346-bed0-97c7f21d16ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:45.924971 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652dcb09-b110-4346-bed0-97c7f21d16ba None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:45.933071 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 627/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:35:45 2022] GET /identity/v3/role_assignments?user.id=6e74dbb121d749749cf5f9d9d7221008&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=7764573472d34ecc8bedf7a9c7935170 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:48.107364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c92be0-a23c-4bb1-b9de-fbb5055766d2 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:48.108019 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c92be0-a23c-4bb1-b9de-fbb5055766d2 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:48.108019 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c92be0-a23c-4bb1-b9de-fbb5055766d2 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:48.109203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 620/1247] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:35:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:35:48.114531 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbce91-e943-4577-acbe-0b48f5b55a04 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:48.115220 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbce91-e943-4577-acbe-0b48f5b55a04 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:48.115220 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbce91-e943-4577-acbe-0b48f5b55a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:48.125099 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:48.138738 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-62cbce91-e943-4577-acbe-0b48f5b55a04 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:35:48.139945 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-62cbce91-e943-4577-acbe-0b48f5b55a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:35:48.215393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 628/1248] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:35:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:35:48.240303 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c92be0-a23c-4bb1-b9de-fbb5055766d2 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:48.259418 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:48.351991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c92be0-a23c-4bb1-b9de-fbb5055766d2 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:48.388405 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f94816-836c-4d10-92a3-94e5bdbdf039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:48.389034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f94816-836c-4d10-92a3-94e5bdbdf039 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:48.389324 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f94816-836c-4d10-92a3-94e5bdbdf039 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:48.389760 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f94816-836c-4d10-92a3-94e5bdbdf039 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:48.400560 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f94816-836c-4d10-92a3-94e5bdbdf039 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:35:48.405769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-a7f94816-836c-4d10-92a3-94e5bdbdf039 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:35:48.405769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:48.405769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:48.405769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:48.405769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:48.405769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:48.405769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:48.405769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:48.405769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:48.405769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:48.405769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:48.405769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:48.405769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:35:48.405769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:35:48.405769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:35:48.405769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:35:48.407236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:35:48.407236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:35:48.407236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:35:48.407236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:35:48.407236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:48.407236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:48.407236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:48.407236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:48.407236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:48.407236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:48.407236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:48.407236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:48.407236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:48.407236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:48.407236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:48.407236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:48.408493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:48.408493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:48.408493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:48.408493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:48.408493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:35:48.408493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:35:48.408493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:35:48.408493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:35:48.408493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:35:48.408493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:35:48.408493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:35:48.408493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:35:48.408493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 621/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:35:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:48.413879 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbce91-e943-4577-acbe-0b48f5b55a04 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:48.522269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbce91-e943-4577-acbe-0b48f5b55a04 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:48.552640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:48.557614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246c045-531d-4c36-b41e-eee532ae7e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:48.558223 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246c045-531d-4c36-b41e-eee532ae7e79 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:48.558497 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246c045-531d-4c36-b41e-eee532ae7e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:48.558799 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246c045-531d-4c36-b41e-eee532ae7e79 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:48.559456 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246c045-531d-4c36-b41e-eee532ae7e79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:48.559755 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246c045-531d-4c36-b41e-eee532ae7e79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:48.561119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246c045-531d-4c36-b41e-eee532ae7e79 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:48.569347 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 629/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:35:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:35:48.575260 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f94816-836c-4d10-92a3-94e5bdbdf039 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:48.657906 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:48.683361 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f94816-836c-4d10-92a3-94e5bdbdf039 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:48.720056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfe2b1-4f44-4bfe-a7d3-88841a3c3534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:48.720702 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfe2b1-4f44-4bfe-a7d3-88841a3c3534 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:48.720979 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfe2b1-4f44-4bfe-a7d3-88841a3c3534 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:48.721245 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfe2b1-4f44-4bfe-a7d3-88841a3c3534 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:48.733136 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfe2b1-4f44-4bfe-a7d3-88841a3c3534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:48.734929 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfe2b1-4f44-4bfe-a7d3-88841a3c3534 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:48.755046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-edcfe2b1-4f44-4bfe-a7d3-88841a3c3534 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:48.755046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:48.755046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:48.755046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:48.755046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:48.755046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:48.755046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:48.755046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:48.755046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:48.755046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:48.755046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:48.755046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:48.755046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:35:48.755046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:35:48.755046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:35:48.755046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:35:48.756492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:48.756492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:48.756492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:48.756492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:48.756492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:48.756492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:48.756492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:48.756492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:48.756492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:48.756492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:48.756492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:48.756492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:48.756492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:48.756492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:48.756492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:48.756492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:48.756492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:35:48.757423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:35:48.757423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:48.757423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:35:48.757423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 622/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:48.763412 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246c045-531d-4c36-b41e-eee532ae7e79 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:48.874339 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246c045-531d-4c36-b41e-eee532ae7e79 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:48.909001 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2fa48-5345-4add-8119-dad0fa2db212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:48.909607 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2fa48-5345-4add-8119-dad0fa2db212 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:48.909876 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2fa48-5345-4add-8119-dad0fa2db212 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:48.910144 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2fa48-5345-4add-8119-dad0fa2db212 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:48.911336 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca2fa48-5345-4add-8119-dad0fa2db212 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:48.911626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca2fa48-5345-4add-8119-dad0fa2db212 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:48.912992 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca2fa48-5345-4add-8119-dad0fa2db212 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:48.928464 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 630/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:35:48.934246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfe2b1-4f44-4bfe-a7d3-88841a3c3534 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:49.036047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfe2b1-4f44-4bfe-a7d3-88841a3c3534 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:49.063077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:49.071149 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64662aae-9f67-4b95-bc74-3a9ef8a1b7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:49.071783 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64662aae-9f67-4b95-bc74-3a9ef8a1b7e4 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:49.072066 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64662aae-9f67-4b95-bc74-3a9ef8a1b7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:49.072330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64662aae-9f67-4b95-bc74-3a9ef8a1b7e4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:49.080452 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64662aae-9f67-4b95-bc74-3a9ef8a1b7e4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:49.081890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64662aae-9f67-4b95-bc74-3a9ef8a1b7e4 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:49.093406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-64662aae-9f67-4b95-bc74-3a9ef8a1b7e4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:35:49.093406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:49.093406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:49.093406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:49.093406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:49.093406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:49.093406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:49.093406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:49.093406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:49.093406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:49.093406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:49.093406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:49.093406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:35:49.093406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:35:49.093406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:35:49.093406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:35:49.095282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:49.095282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:49.095282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:35:49.095282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:35:49.095282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:35:49.095282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:35:49.095282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:49.095282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:49.095282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:49.095282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:49.095282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:49.095282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:49.095282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:49.095282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:49.095282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:49.095282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:49.095282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:49.095282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:49.096661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:49.096661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:49.096661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:35:49.096661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:35:49.096661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:35:49.096661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:35:49.096661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:35:49.096661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:35:49.096661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:35:49.096661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:35:49.096661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 623/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:35:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:49.101608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2fa48-5345-4add-8119-dad0fa2db212 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:49.157570 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:49.201180 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2fa48-5345-4add-8119-dad0fa2db212 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:49.235523 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1bbcc6-b0f8-4089-9006-456bd2fce8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:49.236181 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bbcc6-b0f8-4089-9006-456bd2fce8d0 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:49.236454 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bbcc6-b0f8-4089-9006-456bd2fce8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:49.236718 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bbcc6-b0f8-4089-9006-456bd2fce8d0 None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:49.237434 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1bbcc6-b0f8-4089-9006-456bd2fce8d0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:49.237726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1bbcc6-b0f8-4089-9006-456bd2fce8d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:49.239341 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1bbcc6-b0f8-4089-9006-456bd2fce8d0 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:49.256974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 631/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Aug 8 16:35:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:49.263016 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64662aae-9f67-4b95-bc74-3a9ef8a1b7e4 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:49.367117 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64662aae-9f67-4b95-bc74-3a9ef8a1b7e4 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:49.401447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f825d12-b0bd-418a-a98d-303c2c35ad33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:49.402050 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f825d12-b0bd-418a-a98d-303c2c35ad33 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:49.402325 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f825d12-b0bd-418a-a98d-303c2c35ad33 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:49.402613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f825d12-b0bd-418a-a98d-303c2c35ad33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:49.465028 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:49.479184 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f825d12-b0bd-418a-a98d-303c2c35ad33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:49.480739 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f825d12-b0bd-418a-a98d-303c2c35ad33 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:49.592827 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 624/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:35:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:35:49.599048 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1bbcc6-b0f8-4089-9006-456bd2fce8d0 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:49.707983 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1bbcc6-b0f8-4089-9006-456bd2fce8d0 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:49.726175 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:49.745429 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba11392-99fe-49f1-a5b1-65a137170a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:49.746063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba11392-99fe-49f1-a5b1-65a137170a67 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:49.746339 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba11392-99fe-49f1-a5b1-65a137170a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:49.746630 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba11392-99fe-49f1-a5b1-65a137170a67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:49.757122 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba11392-99fe-49f1-a5b1-65a137170a67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:49.758588 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba11392-99fe-49f1-a5b1-65a137170a67 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:49.773000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-bba11392-99fe-49f1-a5b1-65a137170a67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:49.773000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:49.773000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:49.773000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:49.773000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:49.773000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:49.773000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:49.773000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:49.773000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:49.773000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:49.773000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:49.773000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:49.773000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:35:49.773000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:35:49.773000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:35:49.773000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:35:49.774145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:49.774145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:49.774145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:49.774145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:49.774145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:49.774145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:49.774145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:49.774145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:49.774145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:49.774145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:49.774145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:49.774145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:49.774145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:49.774145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:49.774145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:49.774145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:49.774145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:35:49.775290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:35:49.775290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:49.775290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:35:49.775290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 632/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:49.780014 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f825d12-b0bd-418a-a98d-303c2c35ad33 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:49.872218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:49.883840 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f825d12-b0bd-418a-a98d-303c2c35ad33 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:49.916924 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b938c6-0bb6-4bab-ab71-dc9732b2ccf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:49.917524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b938c6-0bb6-4bab-ab71-dc9732b2ccf1 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:49.917799 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b938c6-0bb6-4bab-ab71-dc9732b2ccf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:49.918068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b938c6-0bb6-4bab-ab71-dc9732b2ccf1 None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:49.918735 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b938c6-0bb6-4bab-ab71-dc9732b2ccf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:49.919039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b938c6-0bb6-4bab-ab71-dc9732b2ccf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:49.920428 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b938c6-0bb6-4bab-ab71-dc9732b2ccf1 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:49.932313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 625/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:49.938024 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba11392-99fe-49f1-a5b1-65a137170a67 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:50.038734 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba11392-99fe-49f1-a5b1-65a137170a67 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:50.071716 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61295b6-c9c6-448d-a559-279b4a788f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:50.072299 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61295b6-c9c6-448d-a559-279b4a788f7e None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:50.072572 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61295b6-c9c6-448d-a559-279b4a788f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:50.072836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61295b6-c9c6-448d-a559-279b4a788f7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:50.079363 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61295b6-c9c6-448d-a559-279b4a788f7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:50.080834 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61295b6-c9c6-448d-a559-279b4a788f7e None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:50.093773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-e61295b6-c9c6-448d-a559-279b4a788f7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:35:50.093773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:50.093773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:50.093773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:50.093773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:50.093773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:50.093773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:50.093773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:50.093773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:50.093773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:50.093773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:50.093773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:50.093773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:35:50.093773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:35:50.093773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:35:50.094955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:35:50.094955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:50.094955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:50.094955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:50.094955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:50.094955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:50.094955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:50.094955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:50.094955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:50.094955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:50.094955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:50.094955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:50.094955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:50.094955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:50.094955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:50.094955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:50.094955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:50.096084 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:35:50.096084 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:35:50.096084 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:35:50.096084 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:35:50.096084 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:35:50.096084 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:35:50.096084 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:35:50.096084 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:35:50.096084 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 633/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:35:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:50.101790 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b938c6-0bb6-4bab-ab71-dc9732b2ccf1 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:50.203593 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b938c6-0bb6-4bab-ab71-dc9732b2ccf1 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:50.238889 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4e55c-71a5-460a-ab84-b27f8960ac57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:50.239522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4e55c-71a5-460a-ab84-b27f8960ac57 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:50.239794 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4e55c-71a5-460a-ab84-b27f8960ac57 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:50.240060 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4e55c-71a5-460a-ab84-b27f8960ac57 None admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:50.240781 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4e55c-71a5-460a-ab84-b27f8960ac57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:50.241081 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4e55c-71a5-460a-ab84-b27f8960ac57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:50.242532 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4e55c-71a5-460a-ab84-b27f8960ac57 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:50.255195 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 626/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:35:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:50.261402 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61295b6-c9c6-448d-a559-279b4a788f7e None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:50.289761 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:50.363173 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61295b6-c9c6-448d-a559-279b4a788f7e None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:50.400612 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925ac2e-c681-47ab-9041-164a1f412832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:50.401310 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925ac2e-c681-47ab-9041-164a1f412832 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:50.401595 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925ac2e-c681-47ab-9041-164a1f412832 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:50.401865 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925ac2e-c681-47ab-9041-164a1f412832 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:50.402608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925ac2e-c681-47ab-9041-164a1f412832 None admin] RBAC: Adding query filter params (role.id=e578ef831127463d9b48cc89fe20329c, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=6e74dbb121d749749cf5f9d9d7221008) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:50.402921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925ac2e-c681-47ab-9041-164a1f412832 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:50.404379 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925ac2e-c681-47ab-9041-164a1f412832 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:50.409904 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 634/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:35:50 2022] GET /identity/v3/role_assignments?user.id=6e74dbb121d749749cf5f9d9d7221008&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=e578ef831127463d9b48cc89fe20329c => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:52.461927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df35061-6d1d-4fb1-8085-186a60edab55 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:52.462489 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df35061-6d1d-4fb1-8085-186a60edab55 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:52.462489 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df35061-6d1d-4fb1-8085-186a60edab55 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:52.463882 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 627/1261] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:35:52.469643 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee263d87-e8e8-4197-962e-daae43644bd3 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:52.470087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee263d87-e8e8-4197-962e-daae43644bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:52.470181 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee263d87-e8e8-4197-962e-daae43644bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:52.488880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-ee263d87-e8e8-4197-962e-daae43644bd3 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:35:52.499509 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ee263d87-e8e8-4197-962e-daae43644bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:35:52.557458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 635/1262] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:35:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:35:52.580607 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df35061-6d1d-4fb1-8085-186a60edab55 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:52.592229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:52.690428 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df35061-6d1d-4fb1-8085-186a60edab55 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:52.731212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1aa942-7145-47d0-97a6-416ceac721eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:52.731929 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1aa942-7145-47d0-97a6-416ceac721eb None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:52.732302 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1aa942-7145-47d0-97a6-416ceac721eb None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:52.732666 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1aa942-7145-47d0-97a6-416ceac721eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:52.747654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1aa942-7145-47d0-97a6-416ceac721eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:35:52.752748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-3f1aa942-7145-47d0-97a6-416ceac721eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:35:52.752748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:52.752748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:52.752748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:52.752748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:52.752748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:52.752748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:52.752748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:52.752748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:52.752748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:52.752748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:52.752748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:52.752748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:35:52.752748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:35:52.752748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:35:52.752748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:35:52.754205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:35:52.754205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:35:52.754205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:35:52.754205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:35:52.754205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:52.754205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:52.754205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:52.754205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:52.754205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:52.754205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:52.754205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:52.754205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:52.754205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:52.754205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:52.754205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:52.754205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:52.755170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:52.755170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:52.755170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:52.755170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:52.755170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:35:52.755170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:35:52.755170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:35:52.755170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:35:52.755170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:35:52.755170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:35:52.755170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:35:52.755170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:35:52.755170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 628/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:35:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:52.760816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee263d87-e8e8-4197-962e-daae43644bd3 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:52.780375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:52.867895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee263d87-e8e8-4197-962e-daae43644bd3 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:52.906071 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf1932-2f40-449b-b3ab-d43399d40d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:52.906730 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf1932-2f40-449b-b3ab-d43399d40d84 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:52.907008 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf1932-2f40-449b-b3ab-d43399d40d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:52.907278 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf1932-2f40-449b-b3ab-d43399d40d84 None admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:52.907954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf1932-2f40-449b-b3ab-d43399d40d84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:52.908252 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf1932-2f40-449b-b3ab-d43399d40d84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:52.909643 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf1932-2f40-449b-b3ab-d43399d40d84 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:52.919965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 636/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:35:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:52.926839 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1aa942-7145-47d0-97a6-416ceac721eb None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:52.997089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:53.030553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1aa942-7145-47d0-97a6-416ceac721eb None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:53.065853 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c679a9-11ed-456c-9ca0-5c9808e9d93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:53.066543 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c679a9-11ed-456c-9ca0-5c9808e9d93e None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:53.066950 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c679a9-11ed-456c-9ca0-5c9808e9d93e None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:53.067384 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c679a9-11ed-456c-9ca0-5c9808e9d93e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:53.077409 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c679a9-11ed-456c-9ca0-5c9808e9d93e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:53.079399 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c679a9-11ed-456c-9ca0-5c9808e9d93e None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:53.095314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-06c679a9-11ed-456c-9ca0-5c9808e9d93e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:53.095314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:53.095314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:53.095314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:53.095314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:53.095314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:53.095314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:53.095314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:53.095314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:53.095314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:53.095314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:53.095314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:53.095314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:35:53.095314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:35:53.095314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:35:53.095314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:35:53.096524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:53.096524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:53.096524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:53.096524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:53.096524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:53.096524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:53.096524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:53.096524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:53.096524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:53.096524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:53.096524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:53.096524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:53.096524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:53.096524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:53.096524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:53.096524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:53.096524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:35:53.098233 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:35:53.098233 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:53.098233 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:35:53.098233 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 629/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:35:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:53.103183 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf1932-2f40-449b-b3ab-d43399d40d84 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:53.186671 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:53.205466 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf1932-2f40-449b-b3ab-d43399d40d84 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:53.239706 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f899a6c-d020-401b-8a75-ebc5f3b0dc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:53.240308 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f899a6c-d020-401b-8a75-ebc5f3b0dc35 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:53.240584 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f899a6c-d020-401b-8a75-ebc5f3b0dc35 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:53.240853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f899a6c-d020-401b-8a75-ebc5f3b0dc35 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:53.241489 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f899a6c-d020-401b-8a75-ebc5f3b0dc35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:53.241788 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f899a6c-d020-401b-8a75-ebc5f3b0dc35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:53.243351 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f899a6c-d020-401b-8a75-ebc5f3b0dc35 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:53.255560 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 637/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:53.261479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c679a9-11ed-456c-9ca0-5c9808e9d93e None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:53.367664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c679a9-11ed-456c-9ca0-5c9808e9d93e None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:53.388164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:53.402679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51b7fd-63dd-4ab9-90fd-5b106310f7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:53.403306 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51b7fd-63dd-4ab9-90fd-5b106310f7fb None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:53.403579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51b7fd-63dd-4ab9-90fd-5b106310f7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:53.403869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51b7fd-63dd-4ab9-90fd-5b106310f7fb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:53.412367 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e51b7fd-63dd-4ab9-90fd-5b106310f7fb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:53.413801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e51b7fd-63dd-4ab9-90fd-5b106310f7fb None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:53.424869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-8e51b7fd-63dd-4ab9-90fd-5b106310f7fb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:35:53.424869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:53.424869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:53.424869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:53.424869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:53.424869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:53.424869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:53.424869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:53.424869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:53.424869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:53.424869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:53.424869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:53.424869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:35:53.424869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:35:53.424869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:35:53.424869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:35:53.426703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:53.426703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:53.426703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:35:53.426703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:35:53.426703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:35:53.426703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:35:53.426703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:53.426703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:53.426703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:53.426703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:53.426703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:53.426703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:53.426703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:53.426703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:53.426703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:53.426703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:53.426703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:53.426703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:53.427936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:53.427936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:53.427936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:35:53.427936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:35:53.427936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:35:53.427936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:35:53.427936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:35:53.427936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:35:53.427936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:35:53.427936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:35:53.427936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 630/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:35:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:53.432845 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f899a6c-d020-401b-8a75-ebc5f3b0dc35 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:53.536215 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f899a6c-d020-401b-8a75-ebc5f3b0dc35 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:53.570070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8566a5-d3a7-4635-8e6a-47e4a7a39048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:53.570705 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8566a5-d3a7-4635-8e6a-47e4a7a39048 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:53.570983 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8566a5-d3a7-4635-8e6a-47e4a7a39048 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:53.571252 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8566a5-d3a7-4635-8e6a-47e4a7a39048 None admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:53.571971 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8566a5-d3a7-4635-8e6a-47e4a7a39048 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:53.572267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8566a5-d3a7-4635-8e6a-47e4a7a39048 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:53.573751 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8566a5-d3a7-4635-8e6a-47e4a7a39048 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:53.582532 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:53.592351 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 638/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Aug 8 16:35:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:53.599094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51b7fd-63dd-4ab9-90fd-5b106310f7fb None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:53.704158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51b7fd-63dd-4ab9-90fd-5b106310f7fb None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:53.744817 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a80e9-9e31-46de-8896-3561f3e58268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:53.745445 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a80e9-9e31-46de-8896-3561f3e58268 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:53.745721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a80e9-9e31-46de-8896-3561f3e58268 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:53.745996 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a80e9-9e31-46de-8896-3561f3e58268 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:53.756849 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a80e9-9e31-46de-8896-3561f3e58268 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:53.758323 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a80e9-9e31-46de-8896-3561f3e58268 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:53.772971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-757a80e9-9e31-46de-8896-3561f3e58268 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:53.772971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:53.772971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:53.772971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:53.772971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:53.772971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:53.772971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:53.772971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:53.772971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:53.772971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:53.772971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:53.772971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:53.772971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:35:53.772971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:35:53.772971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:35:53.772971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:35:53.774232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:53.774232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:53.774232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:53.774232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:53.774232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:53.774232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:53.774232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:53.774232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:53.774232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:53.774232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:53.774232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:53.774232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:53.774232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:53.774232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:53.774232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:53.774232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:53.774232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:35:53.775486 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:35:53.775486 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:53.775486 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:35:53.775486 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 631/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:35:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:53.780002 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8566a5-d3a7-4635-8e6a-47e4a7a39048 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:53.882448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8566a5-d3a7-4635-8e6a-47e4a7a39048 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:53.917741 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3895959-ef20-4062-8cd7-fead36edd791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:53.918338 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3895959-ef20-4062-8cd7-fead36edd791 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:53.918659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3895959-ef20-4062-8cd7-fead36edd791 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:53.918934 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3895959-ef20-4062-8cd7-fead36edd791 None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:53.919590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3895959-ef20-4062-8cd7-fead36edd791 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:53.919886 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3895959-ef20-4062-8cd7-fead36edd791 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:53.921266 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3895959-ef20-4062-8cd7-fead36edd791 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:53.933342 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 639/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:53.939122 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a80e9-9e31-46de-8896-3561f3e58268 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:53.972460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:54.041192 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a80e9-9e31-46de-8896-3561f3e58268 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:54.076358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a884f60-9a1b-4b03-aa48-e9ed37343934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:54.076973 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a884f60-9a1b-4b03-aa48-e9ed37343934 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:54.077248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a884f60-9a1b-4b03-aa48-e9ed37343934 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:54.077513 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a884f60-9a1b-4b03-aa48-e9ed37343934 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:54.087475 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a884f60-9a1b-4b03-aa48-e9ed37343934 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:54.087475 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a884f60-9a1b-4b03-aa48-e9ed37343934 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:54.097589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-6a884f60-9a1b-4b03-aa48-e9ed37343934 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:35:54.097589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:54.097589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:54.097589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:54.097589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:54.097589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:54.097589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:54.097589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:54.097589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:54.097589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:54.097589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:54.097589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:54.097589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:35:54.097589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:35:54.097589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:35:54.099038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:35:54.099038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:54.099038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:54.099038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:54.099038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:54.099038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:54.099038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:54.099038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:54.099038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:54.099038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:54.099038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:54.099038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:54.099038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:54.099038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:54.099038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:54.099038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:54.099038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:54.100382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:35:54.100382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:35:54.100382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:35:54.100382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:35:54.100382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:35:54.100382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:35:54.100382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:35:54.100382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:35:54.100382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 632/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:35:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:54.105303 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3895959-ef20-4062-8cd7-fead36edd791 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:54.164319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:54.206249 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3895959-ef20-4062-8cd7-fead36edd791 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:54.241080 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635d139-1e30-4fd9-b1ab-db8ec445ea57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:54.241689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635d139-1e30-4fd9-b1ab-db8ec445ea57 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:54.241967 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635d139-1e30-4fd9-b1ab-db8ec445ea57 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:54.242236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635d139-1e30-4fd9-b1ab-db8ec445ea57 None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:54.243009 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e635d139-1e30-4fd9-b1ab-db8ec445ea57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:54.243310 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e635d139-1e30-4fd9-b1ab-db8ec445ea57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:54.244850 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e635d139-1e30-4fd9-b1ab-db8ec445ea57 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:54.257641 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 640/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:35:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:54.263834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a884f60-9a1b-4b03-aa48-e9ed37343934 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:54.363031 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:54.372373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a884f60-9a1b-4b03-aa48-e9ed37343934 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:54.405978 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d6bc7-9dc2-4867-a725-cf8ef45c3410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:54.406666 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d6bc7-9dc2-4867-a725-cf8ef45c3410 None admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:54.406944 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d6bc7-9dc2-4867-a725-cf8ef45c3410 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:54.407211 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d6bc7-9dc2-4867-a725-cf8ef45c3410 None admin] PATH_INFO: `/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/6e74dbb121d749749cf5f9d9d7221008/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:54.409366 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75d6bc7-9dc2-4867-a725-cf8ef45c3410 None admin] RBAC: Authorizing `identity:create_grant(project_id=56ab2fc79d964f4d86edefb4d1b2c993, user_id=6e74dbb121d749749cf5f9d9d7221008, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:54.411783 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75d6bc7-9dc2-4867-a725-cf8ef45c3410 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:54.559102 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 633/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Aug 8 16:35:54 2022] PUT /identity/v3/projects/56ab2fc79d964f4d86edefb4d1b2c993/users/6e74dbb121d749749cf5f9d9d7221008/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:35:56.585673 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd4e9e-e3cb-49a5-be5a-1b018224594e None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:56.586318 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd4e9e-e3cb-49a5-be5a-1b018224594e None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:56.586318 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd4e9e-e3cb-49a5-be5a-1b018224594e None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:56.587581 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 641/1274] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:35:56.593064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff89b3-3cc7-4cd3-a6b3-3a8fffb3f461 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:56.593506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff89b3-3cc7-4cd3-a6b3-3a8fffb3f461 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:56.593662 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff89b3-3cc7-4cd3-a6b3-3a8fffb3f461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:56.613015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-adff89b3-3cc7-4cd3-a6b3-3a8fffb3f461 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:35:56.614364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-adff89b3-3cc7-4cd3-a6b3-3a8fffb3f461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:35:56.686077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 634/1275] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:35:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:35:56.710397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd4e9e-e3cb-49a5-be5a-1b018224594e None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:56.814448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd4e9e-e3cb-49a5-be5a-1b018224594e None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:56.832477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:56.849737 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1fee0-dd15-4b72-ae63-0b3f9bef5b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:56.850287 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1fee0-dd15-4b72-ae63-0b3f9bef5b73 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:56.850584 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1fee0-dd15-4b72-ae63-0b3f9bef5b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:56.850861 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1fee0-dd15-4b72-ae63-0b3f9bef5b73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:56.860698 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1fee0-dd15-4b72-ae63-0b3f9bef5b73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:35:56.865564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-5aa1fee0-dd15-4b72-ae63-0b3f9bef5b73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:35:56.865564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:56.865564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:56.865564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:56.865564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:56.865564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:56.865564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:56.865564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:56.865564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:56.865564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:56.865564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:56.865564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:56.865564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:35:56.865564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:35:56.865564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:35:56.865564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:35:56.866599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:35:56.866599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:35:56.866599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:35:56.866599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:35:56.866599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:56.866599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:56.866599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:56.866599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:56.866599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:56.866599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:56.866599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:56.866599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:56.866599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:56.866599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:56.866599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:56.866599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:56.867576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:56.867576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:56.867576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:56.867576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:56.867576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:35:56.867576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:35:56.867576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:35:56.867576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:35:56.867576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:35:56.867576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:35:56.867576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:35:56.867576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:35:56.867576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 642/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Aug 8 16:35:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:56.873004 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff89b3-3cc7-4cd3-a6b3-3a8fffb3f461 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:56.959853 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:56.978321 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff89b3-3cc7-4cd3-a6b3-3a8fffb3f461 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:57.014402 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d073b-f043-4c12-8ee8-703ec99fd014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:57.015020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d073b-f043-4c12-8ee8-703ec99fd014 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:57.015294 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d073b-f043-4c12-8ee8-703ec99fd014 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:57.015558 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d073b-f043-4c12-8ee8-703ec99fd014 None admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:57.016201 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d073b-f043-4c12-8ee8-703ec99fd014 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:57.016496 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d073b-f043-4c12-8ee8-703ec99fd014 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:57.017853 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d073b-f043-4c12-8ee8-703ec99fd014 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:57.026802 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 635/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Aug 8 16:35:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:57.032649 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1fee0-dd15-4b72-ae63-0b3f9bef5b73 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:57.149697 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1fee0-dd15-4b72-ae63-0b3f9bef5b73 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:57.183551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652768d7-0449-4870-923e-83e1ea047f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:57.184172 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652768d7-0449-4870-923e-83e1ea047f9d None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:57.184444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652768d7-0449-4870-923e-83e1ea047f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:57.184707 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652768d7-0449-4870-923e-83e1ea047f9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:57.195352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652768d7-0449-4870-923e-83e1ea047f9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:57.196789 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652768d7-0449-4870-923e-83e1ea047f9d None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:57.211683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-652768d7-0449-4870-923e-83e1ea047f9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:57.211683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:57.211683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:57.211683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:57.211683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:57.211683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:57.211683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:57.211683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:57.211683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:57.211683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:57.211683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:57.211683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:57.211683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:35:57.211683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:35:57.211683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:35:57.211683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:35:57.212922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:57.212922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:57.212922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:57.212922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:57.212922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:57.212922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:57.212922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:57.212922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:57.212922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:57.212922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:57.212922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:57.212922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:57.212922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:57.212922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:57.212922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:57.212922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:57.212922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:35:57.213852 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:35:57.213852 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:57.213852 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:35:57.213852 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 643/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:57.219040 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d073b-f043-4c12-8ee8-703ec99fd014 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:57.320995 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d073b-f043-4c12-8ee8-703ec99fd014 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:57.358252 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a19f24-778d-41fc-b498-7faad8aab185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:57.358889 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a19f24-778d-41fc-b498-7faad8aab185 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:57.359163 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a19f24-778d-41fc-b498-7faad8aab185 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:57.359428 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a19f24-778d-41fc-b498-7faad8aab185 None admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:57.360053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a19f24-778d-41fc-b498-7faad8aab185 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:57.360349 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a19f24-778d-41fc-b498-7faad8aab185 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:57.361703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a19f24-778d-41fc-b498-7faad8aab185 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:57.373954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 636/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:57.380540 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652768d7-0449-4870-923e-83e1ea047f9d None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:57.413301 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:57.483644 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652768d7-0449-4870-923e-83e1ea047f9d None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:57.518402 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad00fc8-be60-4492-938c-98a4b97a1926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:57.519160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad00fc8-be60-4492-938c-98a4b97a1926 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:57.520559 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad00fc8-be60-4492-938c-98a4b97a1926 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:57.520834 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad00fc8-be60-4492-938c-98a4b97a1926 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:57.528343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad00fc8-be60-4492-938c-98a4b97a1926 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:57.529792 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad00fc8-be60-4492-938c-98a4b97a1926 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:57.541601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-bad00fc8-be60-4492-938c-98a4b97a1926 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:35:57.541601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:57.541601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:57.541601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:57.541601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:57.541601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:57.541601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:57.541601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:57.541601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:57.541601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:57.541601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:57.541601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:57.541601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:35:57.541601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:35:57.541601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:35:57.541601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:35:57.542821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:57.542821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:57.542821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:35:57.542821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:35:57.542821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:35:57.542821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:35:57.542821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:57.542821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:57.542821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:57.542821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:57.542821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:57.542821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:57.542821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:57.542821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:57.542821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:57.542821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:57.542821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:57.542821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:57.544057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:57.544057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:57.544057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:35:57.544057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:35:57.544057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:35:57.544057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:35:57.544057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:35:57.544057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:35:57.544057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:35:57.544057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:35:57.544057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 644/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:35:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:57.548710 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a19f24-778d-41fc-b498-7faad8aab185 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:57.553222 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:57.648065 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a19f24-778d-41fc-b498-7faad8aab185 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:57.683667 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72018168-6c77-4b43-83f3-92e35ec8bbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:57.684192 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72018168-6c77-4b43-83f3-92e35ec8bbd8 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:57.684478 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72018168-6c77-4b43-83f3-92e35ec8bbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:57.684746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72018168-6c77-4b43-83f3-92e35ec8bbd8 None admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:57.685468 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72018168-6c77-4b43-83f3-92e35ec8bbd8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:57.685772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72018168-6c77-4b43-83f3-92e35ec8bbd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:57.687242 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72018168-6c77-4b43-83f3-92e35ec8bbd8 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:57.704136 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 637/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Aug 8 16:35:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:57.710887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad00fc8-be60-4492-938c-98a4b97a1926 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:57.804807 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:57.815064 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad00fc8-be60-4492-938c-98a4b97a1926 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:57.849519 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b405f-ea69-4e5a-ab3a-da4ec1eec99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:57.850129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b405f-ea69-4e5a-ab3a-da4ec1eec99b None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:57.850405 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b405f-ea69-4e5a-ab3a-da4ec1eec99b None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:57.850712 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b405f-ea69-4e5a-ab3a-da4ec1eec99b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:57.916496 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b405f-ea69-4e5a-ab3a-da4ec1eec99b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:57.918000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b405f-ea69-4e5a-ab3a-da4ec1eec99b None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:58.024652 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:58.030169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 645/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Aug 8 16:35:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:35:58.040034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72018168-6c77-4b43-83f3-92e35ec8bbd8 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:58.104211 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:58.143037 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72018168-6c77-4b43-83f3-92e35ec8bbd8 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:58.177711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d947ff23-ac7b-4add-ba5c-054e80ff851e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:58.178348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947ff23-ac7b-4add-ba5c-054e80ff851e None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:58.178669 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947ff23-ac7b-4add-ba5c-054e80ff851e None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:58.178938 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947ff23-ac7b-4add-ba5c-054e80ff851e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:58.189290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d947ff23-ac7b-4add-ba5c-054e80ff851e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:58.190764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d947ff23-ac7b-4add-ba5c-054e80ff851e None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:58.205376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-d947ff23-ac7b-4add-ba5c-054e80ff851e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:58.205376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:58.205376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:58.205376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:58.205376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:58.205376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:58.205376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:58.205376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:58.205376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:58.205376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:58.205376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:58.205376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:58.205376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:35:58.205376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:35:58.205376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:35:58.205376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:35:58.215807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:58.215807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:58.215807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:58.215807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:58.215807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:58.215807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:58.215807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:58.215807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:58.215807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:58.215807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:58.215807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:58.215807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:58.215807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:58.215807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:58.215807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:58.215807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:58.215807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:35:58.217298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:35:58.217298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:35:58.217298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:35:58.217298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 638/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:35:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:35:58.217612 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b405f-ea69-4e5a-ab3a-da4ec1eec99b None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:58.313735 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b405f-ea69-4e5a-ab3a-da4ec1eec99b None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:58.349396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc61e0-d11c-4a1a-a500-21cee4ba703d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:58.357798 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc61e0-d11c-4a1a-a500-21cee4ba703d None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:58.357798 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc61e0-d11c-4a1a-a500-21cee4ba703d None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:58.357798 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc61e0-d11c-4a1a-a500-21cee4ba703d None admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:58.357798 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc61e0-d11c-4a1a-a500-21cee4ba703d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:58.357798 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc61e0-d11c-4a1a-a500-21cee4ba703d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:58.357798 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc61e0-d11c-4a1a-a500-21cee4ba703d None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:58.364312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 646/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Aug 8 16:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:58.370047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d947ff23-ac7b-4add-ba5c-054e80ff851e None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:58.471563 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d947ff23-ac7b-4add-ba5c-054e80ff851e None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:58.488145 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:58.510020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd03ee-0506-4f7a-aca1-b2fa3f7984cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:58.512630 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd03ee-0506-4f7a-aca1-b2fa3f7984cc None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:58.512630 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd03ee-0506-4f7a-aca1-b2fa3f7984cc None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:58.512630 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd03ee-0506-4f7a-aca1-b2fa3f7984cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:58.519097 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd03ee-0506-4f7a-aca1-b2fa3f7984cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:58.521452 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd03ee-0506-4f7a-aca1-b2fa3f7984cc None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:58.534860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application [None req-23dd03ee-0506-4f7a-aca1-b2fa3f7984cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:35:58.534860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:35:58.534860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:35:58.534860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:35:58.534860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:35:58.534860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:35:58.534860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:35:58.534860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:35:58.534860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:35:58.534860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:35:58.534860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:35:58.534860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:35:58.534860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:35:58.534860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:35:58.534860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:35:58.536711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:35:58.536711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:35:58.536711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:35:58.536711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:35:58.536711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:35:58.536711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:35:58.536711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:35:58.536711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:35:58.536711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application with Lock( Aug 08 16:35:58.536711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:35:58.536711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:35:58.536711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:35:58.536711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:35:58.536711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:35:58.536711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:35:58.536711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:35:58.536711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:35:58.537809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:35:58.537809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:35:58.537809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:35:58.537809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:35:58.537809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:35:58.537809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:35:58.537809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:35:58.537809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: ERROR keystone.server.flask.application Aug 08 16:35:58.537809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 639/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Aug 8 16:35:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:35:58.542787 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc61e0-d11c-4a1a-a500-21cee4ba703d None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:58.595311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:35:58.645075 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc61e0-d11c-4a1a-a500-21cee4ba703d None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:58.683279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab8fec-dd3c-4199-85e5-d566a67b3b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:58.683888 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab8fec-dd3c-4199-85e5-d566a67b3b4e None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:58.684163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab8fec-dd3c-4199-85e5-d566a67b3b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:58.684429 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab8fec-dd3c-4199-85e5-d566a67b3b4e None admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:58.685119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab8fec-dd3c-4199-85e5-d566a67b3b4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:58.685416 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab8fec-dd3c-4199-85e5-d566a67b3b4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:58.686857 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab8fec-dd3c-4199-85e5-d566a67b3b4e None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:58.699341 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 647/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Aug 8 16:35:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:35:58.705536 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd03ee-0506-4f7a-aca1-b2fa3f7984cc None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:35:58.811352 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd03ee-0506-4f7a-aca1-b2fa3f7984cc None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:35:58.848680 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54539556-3935-42d8-9192-3ea140586f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:35:58.849282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54539556-3935-42d8-9192-3ea140586f36 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:35:58.849561 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54539556-3935-42d8-9192-3ea140586f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:35:58.849824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54539556-3935-42d8-9192-3ea140586f36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:35:58.850537 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54539556-3935-42d8-9192-3ea140586f36 None admin] RBAC: Adding query filter params (role.id=e578ef831127463d9b48cc89fe20329c, scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993, user.id=6e74dbb121d749749cf5f9d9d7221008) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:35:58.850860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54539556-3935-42d8-9192-3ea140586f36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:35:58.852293 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54539556-3935-42d8-9192-3ea140586f36 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:35:58.858182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 640/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Aug 8 16:35:58 2022] GET /identity/v3/role_assignments?user.id=6e74dbb121d749749cf5f9d9d7221008&scope.project.id=56ab2fc79d964f4d86edefb4d1b2c993&role.id=e578ef831127463d9b48cc89fe20329c => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:36:00.953624 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d2e81-babb-4a93-ba40-56adc4d9d2e1 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:36:00.954153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d2e81-babb-4a93-ba40-56adc4d9d2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:36:00.954260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d2e81-babb-4a93-ba40-56adc4d9d2e1 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:36:00.955670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 648/1288] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:36:00.961060 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61383d0-e7e6-4e77-8d51-52562e1f4247 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:36:00.961509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61383d0-e7e6-4e77-8d51-52562e1f4247 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:36:00.961613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61383d0-e7e6-4e77-8d51-52562e1f4247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:36:00.980465 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-d61383d0-e7e6-4e77-8d51-52562e1f4247 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:36:00.981662 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d61383d0-e7e6-4e77-8d51-52562e1f4247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:36:01.011707 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:36:01.049205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 641/1289] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:36:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:36:01.072646 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d2e81-babb-4a93-ba40-56adc4d9d2e1 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:36:01.177941 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d2e81-babb-4a93-ba40-56adc4d9d2e1 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:36:01.189887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:36:01.216429 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1530cbb1-005d-4925-a0eb-a47288042aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:36:01.217059 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530cbb1-005d-4925-a0eb-a47288042aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:36:01.217330 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530cbb1-005d-4925-a0eb-a47288042aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:36:01.217593 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530cbb1-005d-4925-a0eb-a47288042aa7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:36:01.218169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1530cbb1-005d-4925-a0eb-a47288042aa7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:36:01.219570 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1530cbb1-005d-4925-a0eb-a47288042aa7 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:36:01.229659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-1530cbb1-005d-4925-a0eb-a47288042aa7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 16:36:01.229659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:36:01.229659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:36:01.229659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:36:01.229659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:36:01.229659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:36:01.229659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:36:01.229659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:36:01.229659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 16:36:01.229659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:36:01.229659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:36:01.229659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:36:01.229659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:36:01.229659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:36:01.229659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:36:01.229659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:36:01.229659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:36:01.231119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:36:01.231119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:36:01.231119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:36:01.231119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:36:01.231119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:36:01.231119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:36:01.231119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:36:01.231119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:36:01.231119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:36:01.231119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:36:01.231119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:36:01.231119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:36:01.231119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:36:01.231119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:36:01.231119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:36:01.231119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:36:01.232323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:36:01.232323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:36:01.232323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:36:01.232323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:36:01.232323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:36:01.232323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:36:01.232323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:36:01.232323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:36:01.232323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:36:01.232323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:36:01.232323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:36:01.232323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:36:01.232323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 16:36:01.232323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:36:01.232323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 649/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Aug 8 16:36:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:36:01.237290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61383d0-e7e6-4e77-8d51-52562e1f4247 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:36:01.337807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61383d0-e7e6-4e77-8d51-52562e1f4247 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:36:01.372432 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b15b53-c520-4a7f-b672-93ad5e19b94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:36:01.373042 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b15b53-c520-4a7f-b672-93ad5e19b94c None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:36:01.373317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b15b53-c520-4a7f-b672-93ad5e19b94c None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:36:01.373582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b15b53-c520-4a7f-b672-93ad5e19b94c None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:36:01.374251 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b15b53-c520-4a7f-b672-93ad5e19b94c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:36:01.374554 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b15b53-c520-4a7f-b672-93ad5e19b94c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:36:01.375952 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b15b53-c520-4a7f-b672-93ad5e19b94c None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:36:01.381705 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 642/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Aug 8 16:36:01 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:36:01.389062 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1530cbb1-005d-4925-a0eb-a47288042aa7 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:36:01.488242 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1530cbb1-005d-4925-a0eb-a47288042aa7 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:36:01.522195 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e531e-1498-4e65-98c4-dd9d7732e729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:36:01.522876 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e531e-1498-4e65-98c4-dd9d7732e729 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:36:01.523208 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e531e-1498-4e65-98c4-dd9d7732e729 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:36:01.523688 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e531e-1498-4e65-98c4-dd9d7732e729 None admin] PATH_INFO: `/v3/services` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:36:01.524424 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e531e-1498-4e65-98c4-dd9d7732e729 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:36:01.524719 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e531e-1498-4e65-98c4-dd9d7732e729 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:36:01.526096 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e531e-1498-4e65-98c4-dd9d7732e729 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:36:01.530795 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 650/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Aug 8 16:36:01 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:36:03.543809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0564a-c848-4de3-8826-0d0cf86b2146 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:36:03.544396 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0564a-c848-4de3-8826-0d0cf86b2146 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:36:03.544396 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0564a-c848-4de3-8826-0d0cf86b2146 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:36:03.545754 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 643/1293] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:36:03.551612 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc059d6-a43b-4fd6-8162-635a09aaa52f None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:36:03.552154 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc059d6-a43b-4fd6-8162-635a09aaa52f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:36:03.552154 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc059d6-a43b-4fd6-8162-635a09aaa52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:36:03.571022 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-3bc059d6-a43b-4fd6-8162-635a09aaa52f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:36:03.574808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-3bc059d6-a43b-4fd6-8162-635a09aaa52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:36:03.606184 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:36:03.647564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 651/1294] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:36:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:36:03.671563 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0564a-c848-4de3-8826-0d0cf86b2146 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:36:03.709405 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:36:03.793146 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0564a-c848-4de3-8826-0d0cf86b2146 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:36:03.832797 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc663f-33e2-497e-8d14-c359b9e424ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:36:03.832797 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc663f-33e2-497e-8d14-c359b9e424ad None admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:36:03.832797 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc663f-33e2-497e-8d14-c359b9e424ad None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:36:03.832797 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc663f-33e2-497e-8d14-c359b9e424ad None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:36:03.833220 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc663f-33e2-497e-8d14-c359b9e424ad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:36:03.834353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc663f-33e2-497e-8d14-c359b9e424ad None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:36:03.921460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 644/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Aug 8 16:36:03 2022] POST /identity/v3/services => generated 246 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:36:06.031874 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fbf56-8892-4029-9bba-c0eb0ba54858 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:36:06.032464 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fbf56-8892-4029-9bba-c0eb0ba54858 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:36:06.032464 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fbf56-8892-4029-9bba-c0eb0ba54858 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:36:06.033744 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 652/1296] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:36:06.039741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e8041-e1d2-45a9-b477-1e950f3eab17 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:36:06.045038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e8041-e1d2-45a9-b477-1e950f3eab17 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:36:06.045038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e8041-e1d2-45a9-b477-1e950f3eab17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:36:06.058954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-2f7e8041-e1d2-45a9-b477-1e950f3eab17 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:36:06.060153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-2f7e8041-e1d2-45a9-b477-1e950f3eab17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:36:06.138232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 645/1297] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:36:06 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:36:06.162687 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460fbf56-8892-4029-9bba-c0eb0ba54858 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:36:06.270912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460fbf56-8892-4029-9bba-c0eb0ba54858 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:36:06.306907 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ce1b0-b5bf-406d-a2b1-3e72d482138e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:36:06.307533 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ce1b0-b5bf-406d-a2b1-3e72d482138e None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:36:06.307805 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ce1b0-b5bf-406d-a2b1-3e72d482138e None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:36:06.308066 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ce1b0-b5bf-406d-a2b1-3e72d482138e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:36:06.308697 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ce1b0-b5bf-406d-a2b1-3e72d482138e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:36:06.310044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ce1b0-b5bf-406d-a2b1-3e72d482138e None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:36:06.319309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-b24ce1b0-b5bf-406d-a2b1-3e72d482138e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 16:36:06.319309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:36:06.319309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:36:06.319309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:36:06.319309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:36:06.319309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:36:06.319309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:36:06.319309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:36:06.319309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 16:36:06.319309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:36:06.319309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:36:06.319309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:36:06.319309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:36:06.319309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:36:06.319309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:36:06.319309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:36:06.319309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:36:06.325555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:36:06.325555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:36:06.325555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:36:06.325555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:36:06.325555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:36:06.325555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:36:06.325555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:36:06.325555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:36:06.325555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:36:06.325555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:36:06.325555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:36:06.325555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:36:06.325555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:36:06.325555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:36:06.325555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:36:06.325555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:36:06.326653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:36:06.326653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:36:06.326653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:36:06.326653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:36:06.326653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:36:06.326653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:36:06.326653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:36:06.326653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:36:06.326653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:36:06.326653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:36:06.326653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:36:06.326653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:36:06.326653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 16:36:06.326653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:36:06.326653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 653/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Aug 8 16:36:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:36:06.327483 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e8041-e1d2-45a9-b477-1e950f3eab17 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:36:06.345813 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:36:06.428315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e8041-e1d2-45a9-b477-1e950f3eab17 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:36:06.464401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a392dfe-a07c-49eb-abdb-0d120d90e427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:36:06.464999 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a392dfe-a07c-49eb-abdb-0d120d90e427 None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:36:06.465268 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a392dfe-a07c-49eb-abdb-0d120d90e427 None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:36:06.465529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a392dfe-a07c-49eb-abdb-0d120d90e427 None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:36:06.466446 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a392dfe-a07c-49eb-abdb-0d120d90e427 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:36:06.466761 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a392dfe-a07c-49eb-abdb-0d120d90e427 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:36:06.468116 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a392dfe-a07c-49eb-abdb-0d120d90e427 None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:36:06.475210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 646/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Aug 8 16:36:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:36:06.481310 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ce1b0-b5bf-406d-a2b1-3e72d482138e None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:36:06.513233 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:36:06.583611 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ce1b0-b5bf-406d-a2b1-3e72d482138e None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:36:06.619248 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecae32-a240-4a83-bd77-c2453d26d12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:36:06.619859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecae32-a240-4a83-bd77-c2453d26d12f None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:36:06.620130 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecae32-a240-4a83-bd77-c2453d26d12f None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:36:06.620397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecae32-a240-4a83-bd77-c2453d26d12f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:36:06.621087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eecae32-a240-4a83-bd77-c2453d26d12f None admin] RBAC: Adding query filter params (interface=public, service_id=97bef2ba17724b0eb6ec381047eab442, region_id=RegionOne) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:36:06.621389 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eecae32-a240-4a83-bd77-c2453d26d12f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:36:06.622809 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eecae32-a240-4a83-bd77-c2453d26d12f None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:36:06.627897 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 654/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Aug 8 16:36:06 2022] GET /identity/v3/endpoints?service_id=97bef2ba17724b0eb6ec381047eab442&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:36:06.633796 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a392dfe-a07c-49eb-abdb-0d120d90e427 None admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:36:06.717890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:36:06.736079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a392dfe-a07c-49eb-abdb-0d120d90e427 None admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:36:06.771041 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed82267-4e4d-452b-9353-247365626aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:36:06.771649 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed82267-4e4d-452b-9353-247365626aac None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:36:06.771930 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed82267-4e4d-452b-9353-247365626aac None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:36:06.772194 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed82267-4e4d-452b-9353-247365626aac None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:36:06.772802 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed82267-4e4d-452b-9353-247365626aac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:36:06.774231 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed82267-4e4d-452b-9353-247365626aac None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:36:06.783949 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 647/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Aug 8 16:36:06 2022] GET /identity/v3/services => generated 2191 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:36:08.838653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f1304-bba6-4d2a-87ad-78364bad7166 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:36:08.839228 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f1304-bba6-4d2a-87ad-78364bad7166 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:36:08.839228 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f1304-bba6-4d2a-87ad-78364bad7166 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:36:08.840555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 655/1302] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:36:08.846313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3692f-a43d-4e08-90d1-892f8da4582a None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:36:08.847496 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3692f-a43d-4e08-90d1-892f8da4582a None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:36:08.847629 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3692f-a43d-4e08-90d1-892f8da4582a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:36:08.868263 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-57a3692f-a43d-4e08-90d1-892f8da4582a None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:36:08.869468 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-57a3692f-a43d-4e08-90d1-892f8da4582a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:36:08.942924 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 648/1303] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:36:08 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:36:08.967058 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f1304-bba6-4d2a-87ad-78364bad7166 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:36:09.071766 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:36:09.078370 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f1304-bba6-4d2a-87ad-78364bad7166 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:36:09.114773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a83e44-053b-4893-9f6b-b92b0870e042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:36:09.115415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a83e44-053b-4893-9f6b-b92b0870e042 None admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:36:09.115682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a83e44-053b-4893-9f6b-b92b0870e042 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:36:09.115944 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a83e44-053b-4893-9f6b-b92b0870e042 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:36:09.116529 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a83e44-053b-4893-9f6b-b92b0870e042 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:36:09.117879 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a83e44-053b-4893-9f6b-b92b0870e042 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:36:09.126608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application [None req-85a83e44-053b-4893-9f6b-b92b0870e042 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 16:36:09.126608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:36:09.126608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:36:09.126608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:36:09.126608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:36:09.126608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:36:09.126608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:36:09.126608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:36:09.126608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 16:36:09.126608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:36:09.126608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:36:09.126608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:36:09.126608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:36:09.126608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:36:09.126608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:36:09.126608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:36:09.126608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:36:09.128311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:36:09.128311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:36:09.128311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:36:09.128311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:36:09.128311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:36:09.128311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:36:09.128311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:36:09.128311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:36:09.128311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:36:09.128311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:36:09.128311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:36:09.128311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:36:09.128311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:36:09.128311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:36:09.128311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:36:09.128311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:36:09.129198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:36:09.129198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application with Lock( Aug 08 16:36:09.129198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:36:09.129198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:36:09.129198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:36:09.129198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:36:09.129198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:36:09.129198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:36:09.129198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:36:09.129198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:36:09.129198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:36:09.129198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:36:09.129198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 16:36:09.129198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: ERROR keystone.server.flask.application Aug 08 16:36:09.129198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 656/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Aug 8 16:36:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:36:09.134843 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3692f-a43d-4e08-90d1-892f8da4582a None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:36:09.218529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:36:09.241663 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3692f-a43d-4e08-90d1-892f8da4582a None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:36:09.276070 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d982f7-e636-40a8-99fb-dee5b581d3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:36:09.276668 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d982f7-e636-40a8-99fb-dee5b581d3bf None admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:36:09.276939 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d982f7-e636-40a8-99fb-dee5b581d3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:36:09.277201 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d982f7-e636-40a8-99fb-dee5b581d3bf None admin] PATH_INFO: `/v3/services` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:36:09.277821 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d982f7-e636-40a8-99fb-dee5b581d3bf None admin] RBAC: Adding query filter params (name=placement) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:36:09.278111 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d982f7-e636-40a8-99fb-dee5b581d3bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:36:09.279512 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d982f7-e636-40a8-99fb-dee5b581d3bf None admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:36:09.284795 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 649/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Aug 8 16:36:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:36:09.291861 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a83e44-053b-4893-9f6b-b92b0870e042 None admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:36:09.399247 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a83e44-053b-4893-9f6b-b92b0870e042 None admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:36:09.434402 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6c07f-4335-4524-a525-195d8d313472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:36:09.435051 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6c07f-4335-4524-a525-195d8d313472 None admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:36:09.435321 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6c07f-4335-4524-a525-195d8d313472 None admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:36:09.435601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6c07f-4335-4524-a525-195d8d313472 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:36:09.436262 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6c07f-4335-4524-a525-195d8d313472 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:36:09.437623 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6c07f-4335-4524-a525-195d8d313472 None admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:36:09.542313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 657/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:36:09 2022] POST /identity/v3/endpoints => generated 331 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:36:19.312581 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=30 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:36:19.537486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:43:37.047592 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565a259-b8fb-49fd-ab3e-5f13b4be10fe None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:37.048076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565a259-b8fb-49fd-ab3e-5f13b4be10fe None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:37.048170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565a259-b8fb-49fd-ab3e-5f13b4be10fe None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:37.055525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758c4f1-e849-4fc5-8847-ccac79aadf0d None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:37.056495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758c4f1-e849-4fc5-8847-ccac79aadf0d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:37.056495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758c4f1-e849-4fc5-8847-ccac79aadf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:37.061285 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 650/1307] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:43:37 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:43:37.150138 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-c758c4f1-e849-4fc5-8847-ccac79aadf0d None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:43:37.151379 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-c758c4f1-e849-4fc5-8847-ccac79aadf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:37.278947 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 658/1308] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:43:37.321875 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5b840-d219-42c8-b8b9-e8e14ae8fde9 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:37.322750 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5b840-d219-42c8-b8b9-e8e14ae8fde9 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:37.324352 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5b840-d219-42c8-b8b9-e8e14ae8fde9 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:37.327419 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 651/1309] 10.222.0.13 () {54 vars in 965 bytes} [Mon Aug 8 16:43:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:43:37.339357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae634c43-7b8b-4558-8db3-0a608d99c202 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:37.339892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae634c43-7b8b-4558-8db3-0a608d99c202 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:37.341518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae634c43-7b8b-4558-8db3-0a608d99c202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:37.442597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-ae634c43-7b8b-4558-8db3-0a608d99c202 None None] MFA Rules not processed for user `9482bf89c1b648c1a91ee00ee5fbba63`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:43:37.443901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ae634c43-7b8b-4558-8db3-0a608d99c202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:37.472156 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:43:37.531313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 659/1310] 10.222.0.13 () {58 vars in 1062 bytes} [Mon Aug 8 16:43:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:43:37.541519 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5b840-d219-42c8-b8b9-e8e14ae8fde9 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:43:37.543571 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-7fa5b840-d219-42c8-b8b9-e8e14ae8fde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:37.670968 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5b840-d219-42c8-b8b9-e8e14ae8fde9 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:43:37.765314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c170f-8670-4004-8f62-39eeb5bbf154 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:43:37.765946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c170f-8670-4004-8f62-39eeb5bbf154 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:37.766220 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c170f-8670-4004-8f62-39eeb5bbf154 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:37.766493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c170f-8670-4004-8f62-39eeb5bbf154 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:37.768221 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-e38c170f-8670-4004-8f62-39eeb5bbf154 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:37.830966 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:43:37.894897 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c170f-8670-4004-8f62-39eeb5bbf154 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:43:37.896270 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c170f-8670-4004-8f62-39eeb5bbf154 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:43:37.915625 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 652/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:43:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:43:40.257559 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766122d1-4d3e-4a1e-b38d-82c4e1a066d5 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:40.258715 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766122d1-4d3e-4a1e-b38d-82c4e1a066d5 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:40.258715 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766122d1-4d3e-4a1e-b38d-82c4e1a066d5 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:40.260195 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 660/1312] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:43:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:43:40.265675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545ac27-8f47-4d95-bba3-b43ea8c8f657 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:40.266109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545ac27-8f47-4d95-bba3-b43ea8c8f657 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:40.266223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545ac27-8f47-4d95-bba3-b43ea8c8f657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:40.338638 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-a545ac27-8f47-4d95-bba3-b43ea8c8f657 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:43:40.339866 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-a545ac27-8f47-4d95-bba3-b43ea8c8f657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:40.439998 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 653/1313] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:43:40.493534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6aace1-4622-4eed-bcb7-9aa60fb1643f None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:40.493820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6aace1-4622-4eed-bcb7-9aa60fb1643f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:40.494147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6aace1-4622-4eed-bcb7-9aa60fb1643f None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:40.495481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 661/1314] 10.222.0.13 () {54 vars in 965 bytes} [Mon Aug 8 16:43:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:43:40.501325 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd698392-37b3-4c11-838f-08421e25b095 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:40.501677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd698392-37b3-4c11-838f-08421e25b095 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:40.502029 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd698392-37b3-4c11-838f-08421e25b095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:40.516492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:43:40.582252 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-dd698392-37b3-4c11-838f-08421e25b095 None None] MFA Rules not processed for user `9482bf89c1b648c1a91ee00ee5fbba63`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:43:40.583508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-dd698392-37b3-4c11-838f-08421e25b095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:40.663071 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 654/1315] 10.222.0.13 () {58 vars in 1062 bytes} [Mon Aug 8 16:43:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:43:40.672168 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6aace1-4622-4eed-bcb7-9aa60fb1643f None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:43:40.673695 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-8f6aace1-4622-4eed-bcb7-9aa60fb1643f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:40.778738 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6aace1-4622-4eed-bcb7-9aa60fb1643f None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:43:40.850280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:43:40.852955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb44b07-445d-4c01-800a-29015a07b9ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:43:40.853560 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb44b07-445d-4c01-800a-29015a07b9ad service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:40.853840 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb44b07-445d-4c01-800a-29015a07b9ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:40.854107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb44b07-445d-4c01-800a-29015a07b9ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:40.855758 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-bcb44b07-445d-4c01-800a-29015a07b9ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:40.966382 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb44b07-445d-4c01-800a-29015a07b9ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:43:40.967876 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb44b07-445d-4c01-800a-29015a07b9ad service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:43:40.986608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 662/1316] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:43:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:43:43.936149 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d46e41-9f88-4d0e-8e35-7f653f601a3e None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:43.936631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d46e41-9f88-4d0e-8e35-7f653f601a3e None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:43.936724 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d46e41-9f88-4d0e-8e35-7f653f601a3e None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:43.938032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 655/1317] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:43:43.943031 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aea82b-6ce8-42db-92a2-49a87eebea42 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:43.943439 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aea82b-6ce8-42db-92a2-49a87eebea42 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:43.943568 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aea82b-6ce8-42db-92a2-49a87eebea42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:44.009659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-b6aea82b-6ce8-42db-92a2-49a87eebea42 None None] MFA Rules not processed for user `db4e226a05124c368e8ad70f2a45b7ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:43:44.010900 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b6aea82b-6ce8-42db-92a2-49a87eebea42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:44.063826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:43:44.110478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 663/1318] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:43:43 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:43:44.151906 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d46e41-9f88-4d0e-8e35-7f653f601a3e None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:43:44.153485 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-56d46e41-9f88-4d0e-8e35-7f653f601a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:44.259023 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d46e41-9f88-4d0e-8e35-7f653f601a3e None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:43:44.262014 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:43:44.340750 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8cd97-bb06-453a-85fd-cf670c6e39a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:43:44.341374 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8cd97-bb06-453a-85fd-cf670c6e39a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:44.341647 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8cd97-bb06-453a-85fd-cf670c6e39a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:44.341912 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8cd97-bb06-453a-85fd-cf670c6e39a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:44.343599 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-45e8cd97-bb06-453a-85fd-cf670c6e39a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:44.451432 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8cd97-bb06-453a-85fd-cf670c6e39a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:43:44.452868 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8cd97-bb06-453a-85fd-cf670c6e39a6 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:43:44.472073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 656/1319] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:43:44 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:43:47.430886 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedef297-e55d-4aff-a994-c064d55266fe None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:47.431434 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedef297-e55d-4aff-a994-c064d55266fe None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:47.431434 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedef297-e55d-4aff-a994-c064d55266fe None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:47.432764 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 664/1320] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:43:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:43:47.438036 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963f10f-0963-49cd-ac5e-2b42f05b89a2 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:47.438461 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963f10f-0963-49cd-ac5e-2b42f05b89a2 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:47.438556 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963f10f-0963-49cd-ac5e-2b42f05b89a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:47.480127 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:43:47.503988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-f963f10f-0963-49cd-ac5e-2b42f05b89a2 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:43:47.505174 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-f963f10f-0963-49cd-ac5e-2b42f05b89a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:47.602423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 657/1321] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:43:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:43:47.641409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedef297-e55d-4aff-a994-c064d55266fe None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:43:47.642969 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-aedef297-e55d-4aff-a994-c064d55266fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:47.749834 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedef297-e55d-4aff-a994-c064d55266fe None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:43:47.822302 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae99ad-6164-4081-86bd-48830295ffcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:43:47.822924 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae99ad-6164-4081-86bd-48830295ffcf service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:47.823198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae99ad-6164-4081-86bd-48830295ffcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:47.823463 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae99ad-6164-4081-86bd-48830295ffcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:47.825040 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-4cae99ad-6164-4081-86bd-48830295ffcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:47.832585 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:43:47.931895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae99ad-6164-4081-86bd-48830295ffcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:43:47.933325 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae99ad-6164-4081-86bd-48830295ffcf service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:43:47.951771 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 665/1322] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:43:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:43:51.093980 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c4294-3db8-463a-8a97-d1120f425ed0 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:51.094519 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c4294-3db8-463a-8a97-d1120f425ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:51.094519 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c4294-3db8-463a-8a97-d1120f425ed0 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:51.096130 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 658/1323] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:43:51.101465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b69d9a-c821-4065-9527-3cba1a6732c4 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:51.101910 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b69d9a-c821-4065-9527-3cba1a6732c4 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:51.102133 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b69d9a-c821-4065-9527-3cba1a6732c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:51.169870 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-e7b69d9a-c821-4065-9527-3cba1a6732c4 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:43:51.171157 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-e7b69d9a-c821-4065-9527-3cba1a6732c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:51.231698 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:43:51.274614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 666/1324] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:43:51.310215 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c4294-3db8-463a-8a97-d1120f425ed0 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:43:51.311789 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-1b8c4294-3db8-463a-8a97-d1120f425ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:51.424771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c4294-3db8-463a-8a97-d1120f425ed0 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:43:51.439101 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:43:51.506824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d3943-3c87-45d0-9d9b-111b8705c83a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:43:51.507493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d3943-3c87-45d0-9d9b-111b8705c83a service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:51.507776 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d3943-3c87-45d0-9d9b-111b8705c83a service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:51.508045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d3943-3c87-45d0-9d9b-111b8705c83a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:51.509926 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-189d3943-3c87-45d0-9d9b-111b8705c83a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:51.638274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d3943-3c87-45d0-9d9b-111b8705c83a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:43:51.639795 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d3943-3c87-45d0-9d9b-111b8705c83a service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:43:51.661416 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 659/1325] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:43:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:43:53.824904 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdea23-222c-4d1c-8686-6c8f1470a8d8 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:53.825474 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdea23-222c-4d1c-8686-6c8f1470a8d8 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:53.825474 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdea23-222c-4d1c-8686-6c8f1470a8d8 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:53.826814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 667/1326] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:43:53.832481 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdd71e-67a6-4d53-8278-03a99238af8a None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:53.832963 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdd71e-67a6-4d53-8278-03a99238af8a None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:53.833057 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdd71e-67a6-4d53-8278-03a99238af8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:53.891506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:43:53.914953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-63cdd71e-67a6-4d53-8278-03a99238af8a None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:43:53.916316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-63cdd71e-67a6-4d53-8278-03a99238af8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:54.025266 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 660/1327] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:43:54.063939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdea23-222c-4d1c-8686-6c8f1470a8d8 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:43:54.066608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-fdfdea23-222c-4d1c-8686-6c8f1470a8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:54.193513 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdea23-222c-4d1c-8686-6c8f1470a8d8 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:43:54.298601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40e49a-7847-4590-9d9b-ea39da020cd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:43:54.299196 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40e49a-7847-4590-9d9b-ea39da020cd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:54.299479 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40e49a-7847-4590-9d9b-ea39da020cd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:54.299753 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40e49a-7847-4590-9d9b-ea39da020cd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:54.302921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-6e40e49a-7847-4590-9d9b-ea39da020cd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:54.331738 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:43:54.458105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40e49a-7847-4590-9d9b-ea39da020cd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:43:54.459619 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40e49a-7847-4590-9d9b-ea39da020cd8 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:43:54.480219 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 668/1328] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:43:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:43:56.593070 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64076c49-735d-45fe-83cf-4289df76b03f None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:56.593634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64076c49-735d-45fe-83cf-4289df76b03f None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:56.593634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64076c49-735d-45fe-83cf-4289df76b03f None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:56.595014 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 661/1329] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:43:56.600720 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b500514-facb-40c5-8b51-af3a7bf8f2bd None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:56.601259 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b500514-facb-40c5-8b51-af3a7bf8f2bd None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:56.601382 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b500514-facb-40c5-8b51-af3a7bf8f2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:56.680172 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-7b500514-facb-40c5-8b51-af3a7bf8f2bd None None] MFA Rules not processed for user `db4e226a05124c368e8ad70f2a45b7ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:43:56.682021 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-7b500514-facb-40c5-8b51-af3a7bf8f2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:56.755678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:43:56.791803 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 669/1330] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:43:56 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:43:56.831384 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64076c49-735d-45fe-83cf-4289df76b03f None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:43:56.833289 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-64076c49-735d-45fe-83cf-4289df76b03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:56.951225 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64076c49-735d-45fe-83cf-4289df76b03f None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:43:56.972981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:43:57.029597 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97249ba-beca-4c24-ad13-2295107c9cfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:43:57.030107 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97249ba-beca-4c24-ad13-2295107c9cfc service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:57.030379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97249ba-beca-4c24-ad13-2295107c9cfc service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:57.030678 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97249ba-beca-4c24-ad13-2295107c9cfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:57.032275 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d97249ba-beca-4c24-ad13-2295107c9cfc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:57.149024 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97249ba-beca-4c24-ad13-2295107c9cfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:43:57.151096 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97249ba-beca-4c24-ad13-2295107c9cfc service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:43:57.170881 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 662/1331] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:43:56 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:43:59.662286 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5fcc1-d404-46a9-b2fe-ed7a744a705d None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:59.662925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5fcc1-d404-46a9-b2fe-ed7a744a705d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:59.662925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5fcc1-d404-46a9-b2fe-ed7a744a705d None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:59.664213 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 670/1332] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:43:59.669609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f593c0f-8111-4e9e-ab73-667a0c319306 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:43:59.670105 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f593c0f-8111-4e9e-ab73-667a0c319306 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:43:59.670226 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f593c0f-8111-4e9e-ab73-667a0c319306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:43:59.741664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:43:59.744971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-6f593c0f-8111-4e9e-ab73-667a0c319306 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:43:59.746185 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-6f593c0f-8111-4e9e-ab73-667a0c319306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:59.844654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 663/1333] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:43:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:43:59.881546 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee5fcc1-d404-46a9-b2fe-ed7a744a705d None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:43:59.883300 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-8ee5fcc1-d404-46a9-b2fe-ed7a744a705d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:43:59.990639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee5fcc1-d404-46a9-b2fe-ed7a744a705d None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:00.072910 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0766a4b-b6f4-4ed9-9076-251ff00cbd69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:00.073534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0766a4b-b6f4-4ed9-9076-251ff00cbd69 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:00.073840 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0766a4b-b6f4-4ed9-9076-251ff00cbd69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:00.074084 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0766a4b-b6f4-4ed9-9076-251ff00cbd69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:00.075844 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-c0766a4b-b6f4-4ed9-9076-251ff00cbd69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:00.106599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:00.191643 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0766a4b-b6f4-4ed9-9076-251ff00cbd69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:00.192993 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0766a4b-b6f4-4ed9-9076-251ff00cbd69 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:00.211388 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 671/1334] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:43:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:44:02.339554 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6583b-f056-4428-b0dd-cf14985ea805 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:02.340087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6583b-f056-4428-b0dd-cf14985ea805 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:02.340087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6583b-f056-4428-b0dd-cf14985ea805 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:02.341369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 664/1335] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:02.346722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af86e1-64bd-44e4-9052-b6b6e4d39635 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:02.347205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af86e1-64bd-44e4-9052-b6b6e4d39635 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:02.347308 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af86e1-64bd-44e4-9052-b6b6e4d39635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:02.415903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-80af86e1-64bd-44e4-9052-b6b6e4d39635 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:02.417244 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-80af86e1-64bd-44e4-9052-b6b6e4d39635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:02.492726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:02.516199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 672/1336] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:44:02.558199 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b6583b-f056-4428-b0dd-cf14985ea805 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:02.559917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-e7b6583b-f056-4428-b0dd-cf14985ea805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:02.670680 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b6583b-f056-4428-b0dd-cf14985ea805 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:02.700771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:02.750103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd05b50-bebe-4649-8a20-169a111ff330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:02.750763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd05b50-bebe-4649-8a20-169a111ff330 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:02.751040 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd05b50-bebe-4649-8a20-169a111ff330 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:02.751310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd05b50-bebe-4649-8a20-169a111ff330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:02.753022 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-2dd05b50-bebe-4649-8a20-169a111ff330 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:02.866901 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd05b50-bebe-4649-8a20-169a111ff330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:02.870590 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd05b50-bebe-4649-8a20-169a111ff330 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:02.888108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 665/1337] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:44:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:44:06.039060 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3ce8c-f78f-4462-849c-4265afb88595 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:06.039723 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3ce8c-f78f-4462-849c-4265afb88595 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:06.039723 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3ce8c-f78f-4462-849c-4265afb88595 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:06.041777 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 673/1338] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:44:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:06.047559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef7371-ad66-4582-9ae7-1382d5f09db6 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:06.048325 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef7371-ad66-4582-9ae7-1382d5f09db6 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:06.048325 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef7371-ad66-4582-9ae7-1382d5f09db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:06.123675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-a7ef7371-ad66-4582-9ae7-1382d5f09db6 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:06.124907 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-a7ef7371-ad66-4582-9ae7-1382d5f09db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:06.134171 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:06.237193 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 666/1339] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:44:06.276452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3ce8c-f78f-4462-849c-4265afb88595 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:06.278012 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-51f3ce8c-f78f-4462-849c-4265afb88595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:06.400557 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3ce8c-f78f-4462-849c-4265afb88595 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:06.485422 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026891b-ca3f-4285-8d98-47e54256f57e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:06.486090 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026891b-ca3f-4285-8d98-47e54256f57e service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:06.486431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026891b-ca3f-4285-8d98-47e54256f57e service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:06.486786 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026891b-ca3f-4285-8d98-47e54256f57e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:06.488620 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-5026891b-ca3f-4285-8d98-47e54256f57e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:06.534513 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:06.621463 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5026891b-ca3f-4285-8d98-47e54256f57e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:06.622952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5026891b-ca3f-4285-8d98-47e54256f57e service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:06.644047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 674/1340] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:44:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:44:11.177317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b9cfa-bc07-4fe8-a807-59b929313f25 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:11.177902 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b9cfa-bc07-4fe8-a807-59b929313f25 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:11.178097 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b9cfa-bc07-4fe8-a807-59b929313f25 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:11.179646 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 667/1341] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:11.186123 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab79d3-d5ca-470b-b4a7-4d5188800b1c None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:11.186655 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab79d3-d5ca-470b-b4a7-4d5188800b1c None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:11.186755 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab79d3-d5ca-470b-b4a7-4d5188800b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:11.258247 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-afab79d3-d5ca-470b-b4a7-4d5188800b1c None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:11.260272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-afab79d3-d5ca-470b-b4a7-4d5188800b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:11.350853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:11.363593 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 675/1342] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:44:11.404726 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b9cfa-bc07-4fe8-a807-59b929313f25 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:11.406268 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-9b3b9cfa-bc07-4fe8-a807-59b929313f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:11.518094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b9cfa-bc07-4fe8-a807-59b929313f25 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:11.558527 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:11.596725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4e54c-9f1b-4f6e-9844-f719b5b229d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:11.597331 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4e54c-9f1b-4f6e-9844-f719b5b229d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:11.597628 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4e54c-9f1b-4f6e-9844-f719b5b229d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:11.597899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4e54c-9f1b-4f6e-9844-f719b5b229d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:11.599626 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-06e4e54c-9f1b-4f6e-9844-f719b5b229d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:11.718072 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4e54c-9f1b-4f6e-9844-f719b5b229d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:11.719560 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4e54c-9f1b-4f6e-9844-f719b5b229d4 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:11.750471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 668/1343] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:44:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:44:14.923297 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22031a3d-aef7-483a-b28c-bc8ac93b959e None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:14.923823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22031a3d-aef7-483a-b28c-bc8ac93b959e None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:14.923823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22031a3d-aef7-483a-b28c-bc8ac93b959e None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:14.925355 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 676/1344] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:14.931123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1431166-1287-426c-ac31-2bf16020a8ee None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:14.931553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1431166-1287-426c-ac31-2bf16020a8ee None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:14.931553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1431166-1287-426c-ac31-2bf16020a8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:15.010659 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-a1431166-1287-426c-ac31-2bf16020a8ee None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:15.011883 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-a1431166-1287-426c-ac31-2bf16020a8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:15.032206 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:15.123264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 669/1345] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:44:15.161912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22031a3d-aef7-483a-b28c-bc8ac93b959e None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:15.163495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-22031a3d-aef7-483a-b28c-bc8ac93b959e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:15.275400 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22031a3d-aef7-483a-b28c-bc8ac93b959e None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:15.350913 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e6ab3-2f0d-41d7-be36-a797a78e62fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:15.351522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e6ab3-2f0d-41d7-be36-a797a78e62fb service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:15.352091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e6ab3-2f0d-41d7-be36-a797a78e62fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:15.352564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e6ab3-2f0d-41d7-be36-a797a78e62fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:15.353846 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-a94e6ab3-2f0d-41d7-be36-a797a78e62fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:15.401414 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:15.476179 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e6ab3-2f0d-41d7-be36-a797a78e62fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:15.477868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e6ab3-2f0d-41d7-be36-a797a78e62fb service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:15.500735 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 677/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:44:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:44:20.315439 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d591ea0-88d2-4b30-872e-2cbd5a3df71d None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:20.315951 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d591ea0-88d2-4b30-872e-2cbd5a3df71d None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:20.316046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d591ea0-88d2-4b30-872e-2cbd5a3df71d None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:20.317347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 670/1347] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:20.323076 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5daa8e-0be8-41a6-b61e-67d9252f758d None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:20.323586 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5daa8e-0be8-41a6-b61e-67d9252f758d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:20.323681 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5daa8e-0be8-41a6-b61e-67d9252f758d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:20.406978 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-4e5daa8e-0be8-41a6-b61e-67d9252f758d None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:20.408210 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-4e5daa8e-0be8-41a6-b61e-67d9252f758d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:20.528100 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:20.531024 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 678/1348] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:44:20.572308 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d591ea0-88d2-4b30-872e-2cbd5a3df71d None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:20.573936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-1d591ea0-88d2-4b30-872e-2cbd5a3df71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:20.728130 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d591ea0-88d2-4b30-872e-2cbd5a3df71d None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:20.795410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:20.828062 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe337b-f4f8-4551-8f99-9ed2595d74ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:20.828672 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe337b-f4f8-4551-8f99-9ed2595d74ad service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:20.828949 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe337b-f4f8-4551-8f99-9ed2595d74ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:20.829212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe337b-f4f8-4551-8f99-9ed2595d74ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:20.830899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-67fe337b-f4f8-4551-8f99-9ed2595d74ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:20.984226 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe337b-f4f8-4551-8f99-9ed2595d74ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:20.987193 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe337b-f4f8-4551-8f99-9ed2595d74ad service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:21.017613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 671/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:44:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:44:26.084870 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9f776-4afc-4773-b10e-bc6679a2c428 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:26.085369 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9f776-4afc-4773-b10e-bc6679a2c428 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:26.085631 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9f776-4afc-4773-b10e-bc6679a2c428 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:26.088661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 679/1350] 10.222.0.13 () {54 vars in 921 bytes} [Mon Aug 8 16:44:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:26.093840 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da90c4-8dec-4a51-8515-d656f89b097a None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:26.094284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da90c4-8dec-4a51-8515-d656f89b097a None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:26.094367 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da90c4-8dec-4a51-8515-d656f89b097a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:26.212374 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-54da90c4-8dec-4a51-8515-d656f89b097a None None] MFA Rules not processed for user `6e74dbb121d749749cf5f9d9d7221008`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:26.213561 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-54da90c4-8dec-4a51-8515-d656f89b097a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:26.233231 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:26.402674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 672/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Aug 8 16:44:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:44:30.692367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173fa67-5a98-444f-bd83-d76e8a9d7696 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:30.693469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173fa67-5a98-444f-bd83-d76e8a9d7696 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:30.693971 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173fa67-5a98-444f-bd83-d76e8a9d7696 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:30.696580 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 680/1352] 10.222.0.13 () {54 vars in 921 bytes} [Mon Aug 8 16:44:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:30.701962 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5afbd6-58d3-4e18-9baa-c39d83586c66 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:30.702491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5afbd6-58d3-4e18-9baa-c39d83586c66 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:30.702491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5afbd6-58d3-4e18-9baa-c39d83586c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:30.806019 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-fe5afbd6-58d3-4e18-9baa-c39d83586c66 None None] MFA Rules not processed for user `6e74dbb121d749749cf5f9d9d7221008`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:30.807176 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-fe5afbd6-58d3-4e18-9baa-c39d83586c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:30.935748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:30.977052 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 673/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Aug 8 16:44:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:44:32.371487 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d863e-29c5-4777-9fe2-20e4ec0e6396 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:32.374604 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d863e-29c5-4777-9fe2-20e4ec0e6396 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:32.374604 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d863e-29c5-4777-9fe2-20e4ec0e6396 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:32.374604 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 681/1354] 10.222.0.13 () {54 vars in 921 bytes} [Mon Aug 8 16:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:32.378777 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e14e9-a750-4c32-af50-520f6c30442c None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:32.379202 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e14e9-a750-4c32-af50-520f6c30442c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:32.379333 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e14e9-a750-4c32-af50-520f6c30442c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:32.461681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-145e14e9-a750-4c32-af50-520f6c30442c None None] MFA Rules not processed for user `6e74dbb121d749749cf5f9d9d7221008`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:32.463453 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-145e14e9-a750-4c32-af50-520f6c30442c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:32.578181 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 674/1355] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Aug 8 16:44:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:44:36.180471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a2867-e93a-46d7-822c-9a0d1218f46c None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:36.181628 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a2867-e93a-46d7-822c-9a0d1218f46c None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:36.182208 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a2867-e93a-46d7-822c-9a0d1218f46c None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:36.184429 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 682/1356] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:44:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:36.189681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fa123-0996-47ed-bbea-a0a68845c859 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:36.190620 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fa123-0996-47ed-bbea-a0a68845c859 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:36.191065 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fa123-0996-47ed-bbea-a0a68845c859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:36.244066 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:36.287425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-198fa123-0996-47ed-bbea-a0a68845c859 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:36.289040 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-198fa123-0996-47ed-bbea-a0a68845c859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:36.430238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 675/1357] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:44:36.487605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db131fc-8689-4180-b9f3-0818fe97325f None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:36.488469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db131fc-8689-4180-b9f3-0818fe97325f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:36.489088 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db131fc-8689-4180-b9f3-0818fe97325f None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:36.490960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 683/1358] 10.222.0.13 () {54 vars in 962 bytes} [Mon Aug 8 16:44:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:36.500531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764ea92-ea0f-42c8-984e-2ead7366dd6c None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:36.501729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764ea92-ea0f-42c8-984e-2ead7366dd6c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:36.502344 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764ea92-ea0f-42c8-984e-2ead7366dd6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:36.619398 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-9764ea92-ea0f-42c8-984e-2ead7366dd6c None None] MFA Rules not processed for user `9daa7489bced467984d39e8b8ac49d13`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:36.620761 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-9764ea92-ea0f-42c8-984e-2ead7366dd6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:36.698352 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:36.790231 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 676/1359] 10.222.0.13 () {58 vars in 1059 bytes} [Mon Aug 8 16:44:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:44:36.800207 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db131fc-8689-4180-b9f3-0818fe97325f None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:36.802853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-5db131fc-8689-4180-b9f3-0818fe97325f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:36.977214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db131fc-8689-4180-b9f3-0818fe97325f None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:37.060575 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:37.109147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1cd6d-5c24-46a5-ab4e-29f0f64461fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:37.109931 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1cd6d-5c24-46a5-ab4e-29f0f64461fc service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:37.110312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1cd6d-5c24-46a5-ab4e-29f0f64461fc service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:37.110721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1cd6d-5c24-46a5-ab4e-29f0f64461fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:37.112528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-1af1cd6d-5c24-46a5-ab4e-29f0f64461fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:37.228620 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1cd6d-5c24-46a5-ab4e-29f0f64461fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:37.230458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1cd6d-5c24-46a5-ab4e-29f0f64461fc service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:37.250626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 684/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:44:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:44:39.778375 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b8aec-d7d3-40c2-b1eb-7e7112411b61 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:39.778375 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b8aec-d7d3-40c2-b1eb-7e7112411b61 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:39.778375 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b8aec-d7d3-40c2-b1eb-7e7112411b61 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:39.783834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 677/1361] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:44:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:39.785511 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a7116-22e8-428a-958b-d34d3d670e82 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:39.785947 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a7116-22e8-428a-958b-d34d3d670e82 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:39.786373 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a7116-22e8-428a-958b-d34d3d670e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:39.877934 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-443a7116-22e8-428a-958b-d34d3d670e82 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:39.879420 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-443a7116-22e8-428a-958b-d34d3d670e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:39.931257 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:40.000894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 685/1362] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:44:40.048056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b8aec-d7d3-40c2-b1eb-7e7112411b61 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:40.049622 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-897b8aec-d7d3-40c2-b1eb-7e7112411b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:40.303566 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b8aec-d7d3-40c2-b1eb-7e7112411b61 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:40.323521 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:40.515346 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d0097c-f1c6-4021-9da2-232c4ba681cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:40.516264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0097c-f1c6-4021-9da2-232c4ba681cb service nova] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:40.516693 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0097c-f1c6-4021-9da2-232c4ba681cb service nova] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:40.517110 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0097c-f1c6-4021-9da2-232c4ba681cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:40.519604 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-b6d0097c-f1c6-4021-9da2-232c4ba681cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:40.644119 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d0097c-f1c6-4021-9da2-232c4ba681cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:40.645973 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d0097c-f1c6-4021-9da2-232c4ba681cb service nova] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:40.672120 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 678/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:44:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:44:44.104323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b929f-ee53-4ffd-9e0e-72ef8fbd16ab None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:44.105761 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b929f-ee53-4ffd-9e0e-72ef8fbd16ab None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:44.106465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b929f-ee53-4ffd-9e0e-72ef8fbd16ab None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:44.109331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 686/1364] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:44:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:44.118397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e9459-564e-45c5-8e6f-ef2ed3fcd3fc None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:44.119480 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e9459-564e-45c5-8e6f-ef2ed3fcd3fc None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:44.119861 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e9459-564e-45c5-8e6f-ef2ed3fcd3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:44.149465 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:44.258264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-6f0e9459-564e-45c5-8e6f-ef2ed3fcd3fc None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:44.260886 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-6f0e9459-564e-45c5-8e6f-ef2ed3fcd3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:44.483326 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 679/1365] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:44:44.568408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd0d06-9042-49a0-8b0a-ea3024493e3d None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:44.569675 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd0d06-9042-49a0-8b0a-ea3024493e3d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:44.571222 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd0d06-9042-49a0-8b0a-ea3024493e3d None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:44.575008 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 687/1366] 10.222.0.13 () {54 vars in 962 bytes} [Mon Aug 8 16:44:44 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:44.584893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fee3dc-2dbc-44e9-87f4-297e71d6830f None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:44.585379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fee3dc-2dbc-44e9-87f4-297e71d6830f None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:44.586378 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fee3dc-2dbc-44e9-87f4-297e71d6830f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:44.746133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-23fee3dc-2dbc-44e9-87f4-297e71d6830f None None] MFA Rules not processed for user `9daa7489bced467984d39e8b8ac49d13`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:44.748165 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-23fee3dc-2dbc-44e9-87f4-297e71d6830f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:44.818049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:44.995092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 680/1367] 10.222.0.13 () {58 vars in 1059 bytes} [Mon Aug 8 16:44:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:44:44.997562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd0d06-9042-49a0-8b0a-ea3024493e3d None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:44.999466 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-22dd0d06-9042-49a0-8b0a-ea3024493e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:45.213015 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd0d06-9042-49a0-8b0a-ea3024493e3d None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:45.256976 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:45.404339 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dcda12-cc60-430b-bcee-493e76ba7379 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:45.405226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dcda12-cc60-430b-bcee-493e76ba7379 service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:45.405511 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dcda12-cc60-430b-bcee-493e76ba7379 service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:45.405816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dcda12-cc60-430b-bcee-493e76ba7379 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:45.407575 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-81dcda12-cc60-430b-bcee-493e76ba7379 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:45.525701 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dcda12-cc60-430b-bcee-493e76ba7379 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:45.527381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dcda12-cc60-430b-bcee-493e76ba7379 service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:45.550631 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 688/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:44:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:44:45.850596 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86127a24-63c4-4b7e-8156-d4cd3e5b9f6c None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:45.852056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86127a24-63c4-4b7e-8156-d4cd3e5b9f6c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:45.852261 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86127a24-63c4-4b7e-8156-d4cd3e5b9f6c None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:45.855225 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 681/1369] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:44:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:45.864938 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5eb5cd-ee3c-4a21-ad45-bb6aed0c89df None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:45.866090 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5eb5cd-ee3c-4a21-ad45-bb6aed0c89df None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:45.867906 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5eb5cd-ee3c-4a21-ad45-bb6aed0c89df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:45.942926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:46.018376 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-af5eb5cd-ee3c-4a21-ad45-bb6aed0c89df None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:46.021517 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-af5eb5cd-ee3c-4a21-ad45-bb6aed0c89df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:46.214780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 689/1370] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:44:46.536838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a527fc-92b0-4a50-ba91-ea05620dcd6f None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:46.537234 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a527fc-92b0-4a50-ba91-ea05620dcd6f None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:46.537431 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a527fc-92b0-4a50-ba91-ea05620dcd6f None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:46.540565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 682/1371] 10.222.0.13 () {54 vars in 964 bytes} [Mon Aug 8 16:44:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:46.546411 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04262fc-1fe4-4f31-887d-8f2943883d8b None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:46.546870 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04262fc-1fe4-4f31-887d-8f2943883d8b None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:46.547044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04262fc-1fe4-4f31-887d-8f2943883d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:46.656952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-e04262fc-1fe4-4f31-887d-8f2943883d8b None None] MFA Rules not processed for user `7bbf0678f34e46e8b56aaeabce955180`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:46.658181 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-e04262fc-1fe4-4f31-887d-8f2943883d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:46.734611 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:46.798943 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 690/1372] 10.222.0.13 () {58 vars in 1061 bytes} [Mon Aug 8 16:44:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:44:46.803098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a527fc-92b0-4a50-ba91-ea05620dcd6f None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:46.805973 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-70a527fc-92b0-4a50-ba91-ea05620dcd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:46.938407 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a527fc-92b0-4a50-ba91-ea05620dcd6f None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:46.991932 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:47.038039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9967e1-8b4a-4774-85bb-e9a51ef09cf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:47.039809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9967e1-8b4a-4774-85bb-e9a51ef09cf3 service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:47.040882 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9967e1-8b4a-4774-85bb-e9a51ef09cf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:47.041078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9967e1-8b4a-4774-85bb-e9a51ef09cf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:47.044182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-be9967e1-8b4a-4774-85bb-e9a51ef09cf3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:47.254350 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9967e1-8b4a-4774-85bb-e9a51ef09cf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:47.258934 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9967e1-8b4a-4774-85bb-e9a51ef09cf3 service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:47.306536 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 683/1373] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:44:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:44:48.236924 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4fbdb-29c0-42d7-b5d7-d942c932c5fb None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:48.237462 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4fbdb-29c0-42d7-b5d7-d942c932c5fb None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:48.237591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4fbdb-29c0-42d7-b5d7-d942c932c5fb None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:48.239026 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 691/1374] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:48.244567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb4702-25e5-4d2c-b062-e8526b9e6109 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:48.245334 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb4702-25e5-4d2c-b062-e8526b9e6109 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:48.245334 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb4702-25e5-4d2c-b062-e8526b9e6109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:48.318900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-e1eb4702-25e5-4d2c-b062-e8526b9e6109 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:48.320736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-e1eb4702-25e5-4d2c-b062-e8526b9e6109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:48.344299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:48.439428 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 684/1375] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:44:48.492483 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4fbdb-29c0-42d7-b5d7-d942c932c5fb None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:48.494241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-c1c4fbdb-29c0-42d7-b5d7-d942c932c5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:48.651811 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4fbdb-29c0-42d7-b5d7-d942c932c5fb None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:48.708783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:48.780306 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb69ce-405d-49de-8eac-3b7ec44e49ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:48.780836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb69ce-405d-49de-8eac-3b7ec44e49ce service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:48.781113 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb69ce-405d-49de-8eac-3b7ec44e49ce service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:48.781380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb69ce-405d-49de-8eac-3b7ec44e49ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:48.783044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-7adb69ce-405d-49de-8eac-3b7ec44e49ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:48.881563 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb69ce-405d-49de-8eac-3b7ec44e49ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:48.883902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb69ce-405d-49de-8eac-3b7ec44e49ce service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:48.904819 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 692/1376] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:44:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:44:51.240582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe0519-2ef3-464b-a33e-f11e9193daa3 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:51.240582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe0519-2ef3-464b-a33e-f11e9193daa3 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:51.240582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe0519-2ef3-464b-a33e-f11e9193daa3 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:51.243214 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 685/1377] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:44:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:51.250200 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cb847-ada2-42a9-9540-1de27af85042 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:51.251095 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cb847-ada2-42a9-9540-1de27af85042 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:51.251377 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cb847-ada2-42a9-9540-1de27af85042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:51.354655 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-397cb847-ada2-42a9-9540-1de27af85042 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:51.356514 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-397cb847-ada2-42a9-9540-1de27af85042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:51.365072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:51.516373 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 693/1378] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:44:51.566268 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe0519-2ef3-464b-a33e-f11e9193daa3 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:51.568130 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-9cbe0519-2ef3-464b-a33e-f11e9193daa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:51.760503 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe0519-2ef3-464b-a33e-f11e9193daa3 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:51.766033 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:51.904872 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a7b4b-e1b0-48f4-a2a0-790deb7d7543 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:51.905480 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a7b4b-e1b0-48f4-a2a0-790deb7d7543 service nova] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:51.905750 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a7b4b-e1b0-48f4-a2a0-790deb7d7543 service nova] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:51.906010 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a7b4b-e1b0-48f4-a2a0-790deb7d7543 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:51.907657 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-506a7b4b-e1b0-48f4-a2a0-790deb7d7543 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:52.035537 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a7b4b-e1b0-48f4-a2a0-790deb7d7543 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:52.037750 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a7b4b-e1b0-48f4-a2a0-790deb7d7543 service nova] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:52.063673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 686/1379] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:44:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:44:54.427898 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044086b-a000-488c-98f9-bf975fc15307 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:54.428435 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044086b-a000-488c-98f9-bf975fc15307 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:54.428435 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044086b-a000-488c-98f9-bf975fc15307 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:54.430038 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 694/1380] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:54.436880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d9797-cadf-4bb0-a6bc-acdf25302334 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:54.437731 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d9797-cadf-4bb0-a6bc-acdf25302334 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:54.438199 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d9797-cadf-4bb0-a6bc-acdf25302334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:54.453269 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:54.522037 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-567d9797-cadf-4bb0-a6bc-acdf25302334 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:54.524236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-567d9797-cadf-4bb0-a6bc-acdf25302334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:54.652286 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 687/1381] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:44:54.698650 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044086b-a000-488c-98f9-bf975fc15307 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:54.700241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-a044086b-a000-488c-98f9-bf975fc15307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:54.876862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:54.889836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044086b-a000-488c-98f9-bf975fc15307 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:55.036417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ceaee-a60c-4171-958f-db611df282fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:55.037077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ceaee-a60c-4171-958f-db611df282fd service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:55.037353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ceaee-a60c-4171-958f-db611df282fd service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:55.037615 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ceaee-a60c-4171-958f-db611df282fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:55.040475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ec2ceaee-a60c-4171-958f-db611df282fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:55.145490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ceaee-a60c-4171-958f-db611df282fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:55.147151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ceaee-a60c-4171-958f-db611df282fd service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:55.160148 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:55.171677 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 695/1382] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:44:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:44:57.739693 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05bc1c-b31d-4dbb-b44f-da238b356e04 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:57.740421 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05bc1c-b31d-4dbb-b44f-da238b356e04 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:57.740421 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05bc1c-b31d-4dbb-b44f-da238b356e04 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:57.742339 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 688/1383] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:44:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:57.748582 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb687ee-e2be-4184-a330-51c9cd91b061 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:57.749053 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb687ee-e2be-4184-a330-51c9cd91b061 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:57.749345 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb687ee-e2be-4184-a330-51c9cd91b061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:57.829116 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-3bb687ee-e2be-4184-a330-51c9cd91b061 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:57.830259 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-3bb687ee-e2be-4184-a330-51c9cd91b061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:57.936754 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 696/1384] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:44:57.981304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05bc1c-b31d-4dbb-b44f-da238b356e04 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:57.983379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-9a05bc1c-b31d-4dbb-b44f-da238b356e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:58.099072 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:58.173549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05bc1c-b31d-4dbb-b44f-da238b356e04 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:58.300039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bfaf2-9ef4-484d-bc73-d9ca38f68f48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:58.300659 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bfaf2-9ef4-484d-bc73-d9ca38f68f48 service nova] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:58.300938 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bfaf2-9ef4-484d-bc73-d9ca38f68f48 service nova] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:58.301203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bfaf2-9ef4-484d-bc73-d9ca38f68f48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:58.302952 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-f15bfaf2-9ef4-484d-bc73-d9ca38f68f48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:58.360113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:58.409429 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15bfaf2-9ef4-484d-bc73-d9ca38f68f48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:58.411674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15bfaf2-9ef4-484d-bc73-d9ca38f68f48 service nova] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:58.439048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 689/1385] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:44:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:45:00.574229 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1848c-9f73-4f72-a306-a2faff747b31 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:00.575010 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1848c-9f73-4f72-a306-a2faff747b31 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:00.575010 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1848c-9f73-4f72-a306-a2faff747b31 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:00.576541 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 697/1386] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:45:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:45:00.582362 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbd03a-8163-4191-8852-0b9a381e9ac1 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:00.583008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbd03a-8163-4191-8852-0b9a381e9ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:00.583008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbd03a-8163-4191-8852-0b9a381e9ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:00.668505 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-2acbd03a-8163-4191-8852-0b9a381e9ac1 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:00.670274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-2acbd03a-8163-4191-8852-0b9a381e9ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:00.796370 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 690/1387] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:45:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:45:00.846975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1848c-9f73-4f72-a306-a2faff747b31 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:45:00.848798 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-2cd1848c-9f73-4f72-a306-a2faff747b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:00.951053 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:01.036388 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1848c-9f73-4f72-a306-a2faff747b31 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:45:01.188667 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd7be2-5b74-4b28-bd54-5dee0d1bb046 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:45:01.189697 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd7be2-5b74-4b28-bd54-5dee0d1bb046 service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:01.190180 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd7be2-5b74-4b28-bd54-5dee0d1bb046 service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:01.190657 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd7be2-5b74-4b28-bd54-5dee0d1bb046 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:01.193283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f9fd7be2-5b74-4b28-bd54-5dee0d1bb046 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:01.239906 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:01.307090 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd7be2-5b74-4b28-bd54-5dee0d1bb046 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:45:01.308828 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd7be2-5b74-4b28-bd54-5dee0d1bb046 service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:45:01.330104 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 698/1388] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:45:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:45:03.449890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5035008-bc17-4d0d-ab88-12ecf4b7c990 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:03.450498 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5035008-bc17-4d0d-ab88-12ecf4b7c990 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:03.450498 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5035008-bc17-4d0d-ab88-12ecf4b7c990 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:03.451908 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 691/1389] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:45:03.457654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc409e-cc27-4645-9c85-ecbf9a9607c7 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:03.458120 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc409e-cc27-4645-9c85-ecbf9a9607c7 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:03.458239 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc409e-cc27-4645-9c85-ecbf9a9607c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:03.529242 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-24dc409e-cc27-4645-9c85-ecbf9a9607c7 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:03.530472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-24dc409e-cc27-4645-9c85-ecbf9a9607c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:03.587013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06ce39-f840-4a45-9c4a-1a8a516cc356 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:03.587300 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06ce39-f840-4a45-9c4a-1a8a516cc356 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:03.587562 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06ce39-f840-4a45-9c4a-1a8a516cc356 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:03.588912 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 692/1390] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:45:03.594212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f69334-ebb2-4a15-ac75-1f1d6204fc53 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:03.594494 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f69334-ebb2-4a15-ac75-1f1d6204fc53 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:03.594797 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f69334-ebb2-4a15-ac75-1f1d6204fc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:03.632178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:03.637357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 699/1391] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:45:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:45:03.667534 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-a8f69334-ebb2-4a15-ac75-1f1d6204fc53 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:03.668167 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-a8f69334-ebb2-4a15-ac75-1f1d6204fc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:03.695198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef5a6d-3bc4-4012-94a9-e638cfc14b9b None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:03.695650 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef5a6d-3bc4-4012-94a9-e638cfc14b9b None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:03.695754 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef5a6d-3bc4-4012-94a9-e638cfc14b9b None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:03.697061 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 700/1392] 10.222.0.13 () {54 vars in 967 bytes} [Mon Aug 8 16:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:45:03.702201 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7008a19-e7e8-4698-961a-628de6c1971a None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:03.702416 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7008a19-e7e8-4698-961a-628de6c1971a None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:03.702720 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7008a19-e7e8-4698-961a-628de6c1971a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:03.719566 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:03.776176 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 693/1393] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:45:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:45:03.791643 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-c7008a19-e7e8-4698-961a-628de6c1971a None None] MFA Rules not processed for user `ea51ef02128a4b63b11049eb0daef298`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:03.792874 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-c7008a19-e7e8-4698-961a-628de6c1971a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:03.823200 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f69334-ebb2-4a15-ac75-1f1d6204fc53 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:45:03.824506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-a8f69334-ebb2-4a15-ac75-1f1d6204fc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:03.878881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 701/1394] 10.222.0.13 () {58 vars in 1064 bytes} [Mon Aug 8 16:45:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:45:03.887271 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7008a19-e7e8-4698-961a-628de6c1971a None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:45:03.895222 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7008a19-e7e8-4698-961a-628de6c1971a None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:45:03.958948 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:03.965117 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f69334-ebb2-4a15-ac75-1f1d6204fc53 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:45:03.992852 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978d1d8-3931-4c2a-a480-47564e4f8b08 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea51ef02128a4b63b11049eb0daef298', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:45:03.993487 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978d1d8-3931-4c2a-a480-47564e4f8b08 service glance] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:03.993765 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978d1d8-3931-4c2a-a480-47564e4f8b08 service glance] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:03.994038 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978d1d8-3931-4c2a-a480-47564e4f8b08 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:03.995973 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-a978d1d8-3931-4c2a-a480-47564e4f8b08 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:04.033544 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:04.105401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c984574f-967a-45ae-8214-1292808ed5d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:45:04.106198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984574f-967a-45ae-8214-1292808ed5d3 service nova] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:04.106588 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984574f-967a-45ae-8214-1292808ed5d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:04.106991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984574f-967a-45ae-8214-1292808ed5d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:04.108777 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-c984574f-967a-45ae-8214-1292808ed5d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:04.134472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a978d1d8-3931-4c2a-a480-47564e4f8b08 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:45:04.136036 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a978d1d8-3931-4c2a-a480-47564e4f8b08 service glance] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:45:04.160892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 702/1395] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:45:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:45:04.213119 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c984574f-967a-45ae-8214-1292808ed5d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:45:04.215051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c984574f-967a-45ae-8214-1292808ed5d3 service nova] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:45:04.233116 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:04.236556 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 694/1396] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:45:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:45:05.116218 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4f0e1-a4d4-4cef-abe1-c7b78a459c8c None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:05.116856 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4f0e1-a4d4-4cef-abe1-c7b78a459c8c None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:05.116955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4f0e1-a4d4-4cef-abe1-c7b78a459c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:05.189788 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-09a4f0e1-a4d4-4cef-abe1-c7b78a459c8c None None] MFA Rules not processed for user `9646662b0b3048f79ad5d2b9602f10a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:05.191034 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-09a4f0e1-a4d4-4cef-abe1-c7b78a459c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:05.283148 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:05.289439 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 703/1397] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Aug 8 16:45:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:45:05.339094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31947cd1-f618-4849-9cb9-b48a83847834 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:05.339669 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31947cd1-f618-4849-9cb9-b48a83847834 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:05.340061 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31947cd1-f618-4849-9cb9-b48a83847834 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:05.342228 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 695/1398] 10.222.0.13 () {54 vars in 943 bytes} [Mon Aug 8 16:45:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:45:05.349393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34899f6-f4ba-4f55-8af5-2c51773d28ec None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:05.349881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34899f6-f4ba-4f55-8af5-2c51773d28ec None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:05.350273 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34899f6-f4ba-4f55-8af5-2c51773d28ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:05.436488 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-b34899f6-f4ba-4f55-8af5-2c51773d28ec None None] MFA Rules not processed for user `c606decb832444e38a194b98f8d15f8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:05.437726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b34899f6-f4ba-4f55-8af5-2c51773d28ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:05.560611 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 704/1399] 10.222.0.13 () {58 vars in 1040 bytes} [Mon Aug 8 16:45:05 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:45:05.571455 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31947cd1-f618-4849-9cb9-b48a83847834 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:45:05.573714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-31947cd1-f618-4849-9cb9-b48a83847834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:05.735922 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31947cd1-f618-4849-9cb9-b48a83847834 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:45:05.840731 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:05.864107 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eda874-6120-42b7-a3c0-d85d13e6be81 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c606decb832444e38a194b98f8d15f8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:45:05.869381 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eda874-6120-42b7-a3c0-d85d13e6be81 service swift] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:05.869381 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eda874-6120-42b7-a3c0-d85d13e6be81 service swift] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:05.869381 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eda874-6120-42b7-a3c0-d85d13e6be81 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:05.869381 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-b2eda874-6120-42b7-a3c0-d85d13e6be81 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:05.982472 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eda874-6120-42b7-a3c0-d85d13e6be81 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:45:05.984290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eda874-6120-42b7-a3c0-d85d13e6be81 service swift] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:45:05.996444 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 696/1400] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Aug 8 16:45:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 16:45:06.551617 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea14eee-d068-4c3f-974f-e7946b72b71d None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:06.552165 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea14eee-d068-4c3f-974f-e7946b72b71d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:06.552165 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea14eee-d068-4c3f-974f-e7946b72b71d None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:06.553755 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 705/1401] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:45:06.559254 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f6c2d-1e8c-4fde-9aa3-a2c80d645c01 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:06.559729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f6c2d-1e8c-4fde-9aa3-a2c80d645c01 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:06.559824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f6c2d-1e8c-4fde-9aa3-a2c80d645c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:06.641088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-703f6c2d-1e8c-4fde-9aa3-a2c80d645c01 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:06.642321 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-703f6c2d-1e8c-4fde-9aa3-a2c80d645c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:06.735772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:06.777870 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 697/1402] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:45:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:45:06.829438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea14eee-d068-4c3f-974f-e7946b72b71d None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:45:06.835050 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-cea14eee-d068-4c3f-974f-e7946b72b71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:06.933232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:07.061088 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea14eee-d068-4c3f-974f-e7946b72b71d None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:45:07.244572 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5bf67e-3a16-409a-b434-94a40d833f6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:45:07.244572 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bf67e-3a16-409a-b434-94a40d833f6f service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:07.244572 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bf67e-3a16-409a-b434-94a40d833f6f service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:07.244572 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bf67e-3a16-409a-b434-94a40d833f6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:07.245820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-eb5bf67e-3a16-409a-b434-94a40d833f6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:07.266513 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:07.380717 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5bf67e-3a16-409a-b434-94a40d833f6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:45:07.382997 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5bf67e-3a16-409a-b434-94a40d833f6f service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:45:07.405811 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 706/1403] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:45:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:45:09.789231 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ee9da-484c-4346-86f7-f2a8d5f55b27 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:09.789231 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ee9da-484c-4346-86f7-f2a8d5f55b27 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:09.789231 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ee9da-484c-4346-86f7-f2a8d5f55b27 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:09.790468 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 698/1404] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:45:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:45:09.795035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c33b87-09c6-4b58-9290-128c7ab3d7d7 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:09.795534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c33b87-09c6-4b58-9290-128c7ab3d7d7 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:09.795639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c33b87-09c6-4b58-9290-128c7ab3d7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:09.876160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-47c33b87-09c6-4b58-9290-128c7ab3d7d7 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:09.877592 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-47c33b87-09c6-4b58-9290-128c7ab3d7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:10.007485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:10.019295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 707/1405] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:45:10.067928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ee9da-484c-4346-86f7-f2a8d5f55b27 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:45:10.069501 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-6a0ee9da-484c-4346-86f7-f2a8d5f55b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:10.281987 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ee9da-484c-4346-86f7-f2a8d5f55b27 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:45:10.359914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:10.432808 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030bf8b-c17c-4c04-b375-2fbc840611f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:45:10.433994 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030bf8b-c17c-4c04-b375-2fbc840611f9 service nova] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:10.434334 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030bf8b-c17c-4c04-b375-2fbc840611f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:10.434735 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030bf8b-c17c-4c04-b375-2fbc840611f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:10.436456 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-b030bf8b-c17c-4c04-b375-2fbc840611f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:10.546129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b030bf8b-c17c-4c04-b375-2fbc840611f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:45:10.547953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b030bf8b-c17c-4c04-b375-2fbc840611f9 service nova] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:45:10.572477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 699/1406] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:45:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:45:12.848047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201fe06f-1321-45a6-95b4-367918ebe07a None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:12.848591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201fe06f-1321-45a6-95b4-367918ebe07a None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:12.848591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201fe06f-1321-45a6-95b4-367918ebe07a None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:12.849867 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 708/1407] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:45:12.855335 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712dbd38-fae0-4d6d-b690-ee77db1734c4 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:12.855824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712dbd38-fae0-4d6d-b690-ee77db1734c4 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:12.856158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712dbd38-fae0-4d6d-b690-ee77db1734c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:12.933016 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-712dbd38-fae0-4d6d-b690-ee77db1734c4 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:12.934260 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-712dbd38-fae0-4d6d-b690-ee77db1734c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:12.957509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:13.064737 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 700/1408] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:45:13.110743 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201fe06f-1321-45a6-95b4-367918ebe07a None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:45:13.112631 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-201fe06f-1321-45a6-95b4-367918ebe07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:13.408496 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201fe06f-1321-45a6-95b4-367918ebe07a None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:45:13.496828 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:13.558324 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470a3c8-35d6-45ae-b166-91f1c50e3ef1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:45:13.558988 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470a3c8-35d6-45ae-b166-91f1c50e3ef1 service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:13.559250 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470a3c8-35d6-45ae-b166-91f1c50e3ef1 service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:13.559526 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470a3c8-35d6-45ae-b166-91f1c50e3ef1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:13.561247 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-2470a3c8-35d6-45ae-b166-91f1c50e3ef1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:13.675305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2470a3c8-35d6-45ae-b166-91f1c50e3ef1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:45:13.676811 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2470a3c8-35d6-45ae-b166-91f1c50e3ef1 service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:45:13.699371 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 709/1409] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:45:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:45:23.062174 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=37 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:23.695741 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=65 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:30.322495 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2b81d-214d-4e35-b951-cd7f87296200 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:30.323105 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2b81d-214d-4e35-b951-cd7f87296200 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:30.323105 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2b81d-214d-4e35-b951-cd7f87296200 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:30.324364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 701/1410] 10.222.0.13 () {54 vars in 918 bytes} [Mon Aug 8 16:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:45:30.329927 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beedf4a-d370-40da-bd98-6062ec48724e None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:30.330817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beedf4a-d370-40da-bd98-6062ec48724e None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:30.330817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beedf4a-d370-40da-bd98-6062ec48724e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:30.411870 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-1beedf4a-d370-40da-bd98-6062ec48724e None None] MFA Rules not processed for user `6e74dbb121d749749cf5f9d9d7221008`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:30.413237 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-1beedf4a-d370-40da-bd98-6062ec48724e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:30.526361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 710/1411] 10.222.0.13 () {58 vars in 1015 bytes} [Mon Aug 8 16:45:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:45:30.542725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8835394-efdd-47ff-adce-8bf35a0dc858 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:30.543082 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8835394-efdd-47ff-adce-8bf35a0dc858 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:30.543361 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8835394-efdd-47ff-adce-8bf35a0dc858 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:30.544713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 702/1412] 10.222.0.13 () {54 vars in 918 bytes} [Mon Aug 8 16:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:45:30.557157 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bc6a6-7218-473f-b0b5-a4945093374a None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:30.557445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bc6a6-7218-473f-b0b5-a4945093374a None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:30.557726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bc6a6-7218-473f-b0b5-a4945093374a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:30.637489 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:30.640824 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-eb2bc6a6-7218-473f-b0b5-a4945093374a None None] MFA Rules not processed for user `9daa7489bced467984d39e8b8ac49d13`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:30.642062 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-eb2bc6a6-7218-473f-b0b5-a4945093374a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:30.777040 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 711/1413] 10.222.0.13 () {58 vars in 1015 bytes} [Mon Aug 8 16:45:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:45:33.370006 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fea81-f2d2-4f71-b96e-84097481517c None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:33.370716 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fea81-f2d2-4f71-b96e-84097481517c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:33.370716 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fea81-f2d2-4f71-b96e-84097481517c None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:33.371907 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 703/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:45:33.377453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5661ad-3d8b-4403-aba3-53d957204422 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:33.377964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5661ad-3d8b-4403-aba3-53d957204422 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:33.377964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5661ad-3d8b-4403-aba3-53d957204422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:33.451103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-dd5661ad-3d8b-4403-aba3-53d957204422 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:33.452321 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-dd5661ad-3d8b-4403-aba3-53d957204422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:33.512320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:33.559529 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 712/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:45:33.604316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0fea81-f2d2-4f71-b96e-84097481517c None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:45:33.605852 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-8b0fea81-f2d2-4f71-b96e-84097481517c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:33.765681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0fea81-f2d2-4f71-b96e-84097481517c None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:45:33.848603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:33.905108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83dcc7-d8f7-40ab-8bcd-295cf47f030e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:45:33.905730 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83dcc7-d8f7-40ab-8bcd-295cf47f030e service nova] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:33.906003 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83dcc7-d8f7-40ab-8bcd-295cf47f030e service nova] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:33.906267 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83dcc7-d8f7-40ab-8bcd-295cf47f030e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:33.908027 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ad83dcc7-d8f7-40ab-8bcd-295cf47f030e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:34.052071 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83dcc7-d8f7-40ab-8bcd-295cf47f030e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:45:34.053648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83dcc7-d8f7-40ab-8bcd-295cf47f030e service nova] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:45:34.075738 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 704/1416] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:45:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:45:35.007813 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e21c0-1adc-4eab-b67f-eba0fde1f0a5 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:35.008392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e21c0-1adc-4eab-b67f-eba0fde1f0a5 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:35.008392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e21c0-1adc-4eab-b67f-eba0fde1f0a5 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:35.009714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 713/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:45:35.016227 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a0614-9ae9-43ad-b14d-ada3b39dd0e5 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:35.016734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a0614-9ae9-43ad-b14d-ada3b39dd0e5 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:35.017152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a0614-9ae9-43ad-b14d-ada3b39dd0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:35.134541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-e64a0614-9ae9-43ad-b14d-ada3b39dd0e5 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:35.136152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-e64a0614-9ae9-43ad-b14d-ada3b39dd0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:35.168211 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:35.258371 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 705/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:45:35.411964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e21c0-1adc-4eab-b67f-eba0fde1f0a5 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:45:35.414831 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-756e21c0-1adc-4eab-b67f-eba0fde1f0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:35.605157 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e21c0-1adc-4eab-b67f-eba0fde1f0a5 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:45:35.650338 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:35.737757 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91023f4e-44d5-44ad-8216-ff1686c26503 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:45:35.738359 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91023f4e-44d5-44ad-8216-ff1686c26503 service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:35.738667 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91023f4e-44d5-44ad-8216-ff1686c26503 service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:35.738940 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91023f4e-44d5-44ad-8216-ff1686c26503 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:35.741096 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-91023f4e-44d5-44ad-8216-ff1686c26503 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:35.841799 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91023f4e-44d5-44ad-8216-ff1686c26503 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:45:35.843446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91023f4e-44d5-44ad-8216-ff1686c26503 service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:45:35.863858 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 714/1419] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:45:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:45:36.931664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477e09f-d9a2-4cda-9c98-c9f52eb22446 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:36.932332 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477e09f-d9a2-4cda-9c98-c9f52eb22446 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:36.932332 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477e09f-d9a2-4cda-9c98-c9f52eb22446 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:36.933583 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 706/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:45:36.939232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7012906-79dd-4dda-b91c-cc071c92de9b None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:36.940016 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7012906-79dd-4dda-b91c-cc071c92de9b None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:36.940016 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7012906-79dd-4dda-b91c-cc071c92de9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:37.005162 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:37.032505 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-b7012906-79dd-4dda-b91c-cc071c92de9b None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:37.034019 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b7012906-79dd-4dda-b91c-cc071c92de9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:37.157910 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 715/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:45:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:45:37.321586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477e09f-d9a2-4cda-9c98-c9f52eb22446 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:45:37.324478 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-9477e09f-d9a2-4cda-9c98-c9f52eb22446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:37.509396 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477e09f-d9a2-4cda-9c98-c9f52eb22446 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:45:37.522641 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:37.637837 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e6391-04e5-4c8f-9802-888df89c7826 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:45:37.638461 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e6391-04e5-4c8f-9802-888df89c7826 service nova] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:37.638771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e6391-04e5-4c8f-9802-888df89c7826 service nova] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:37.639043 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e6391-04e5-4c8f-9802-888df89c7826 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:37.640707 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-0e1e6391-04e5-4c8f-9802-888df89c7826 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:37.743011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e6391-04e5-4c8f-9802-888df89c7826 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:45:37.744729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e6391-04e5-4c8f-9802-888df89c7826 service nova] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:45:37.765831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 707/1422] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:45:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:45:38.720225 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dbfa1-b0c0-4209-80e3-d07698cff38a None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:38.720830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dbfa1-b0c0-4209-80e3-d07698cff38a None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:38.720830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dbfa1-b0c0-4209-80e3-d07698cff38a None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:38.722365 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 716/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:45:38.728902 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199685f-e368-4737-95b2-0dad28b9406e None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:38.733129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199685f-e368-4737-95b2-0dad28b9406e None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:38.733129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199685f-e368-4737-95b2-0dad28b9406e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:38.751665 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:38.823055 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-5199685f-e368-4737-95b2-0dad28b9406e None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:38.824197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-5199685f-e368-4737-95b2-0dad28b9406e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:38.951292 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 708/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:45:39.111447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593dbfa1-b0c0-4209-80e3-d07698cff38a None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:45:39.113465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-593dbfa1-b0c0-4209-80e3-d07698cff38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:39.239895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:39.282270 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593dbfa1-b0c0-4209-80e3-d07698cff38a None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:45:39.411756 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f29067-1fae-49fa-8b02-66986f317767 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:45:39.413382 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f29067-1fae-49fa-8b02-66986f317767 service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:39.413702 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f29067-1fae-49fa-8b02-66986f317767 service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:39.414011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f29067-1fae-49fa-8b02-66986f317767 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:39.415829 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-01f29067-1fae-49fa-8b02-66986f317767 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:39.506977 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:39.522453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f29067-1fae-49fa-8b02-66986f317767 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:45:39.524488 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f29067-1fae-49fa-8b02-66986f317767 service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:45:39.554613 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 717/1425] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:45:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:45:40.506401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131db09-7039-46e1-b276-c1ee839ecd6f None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:40.507134 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131db09-7039-46e1-b276-c1ee839ecd6f None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:40.507390 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131db09-7039-46e1-b276-c1ee839ecd6f None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:40.508976 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 709/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:45:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:45:40.514810 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae4f3e-b26a-49f6-888c-49ddaa8d4bd1 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:40.515356 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae4f3e-b26a-49f6-888c-49ddaa8d4bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:40.515356 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae4f3e-b26a-49f6-888c-49ddaa8d4bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:40.598192 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-9bae4f3e-b26a-49f6-888c-49ddaa8d4bd1 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:40.599457 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-9bae4f3e-b26a-49f6-888c-49ddaa8d4bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:40.702167 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 718/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:45:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:45:40.860584 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4131db09-7039-46e1-b276-c1ee839ecd6f None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:45:40.862754 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-4131db09-7039-46e1-b276-c1ee839ecd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:40.963397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:41.030891 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4131db09-7039-46e1-b276-c1ee839ecd6f None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:45:41.157615 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3a546-6465-468b-87fb-fcc5c803697f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:45:41.158128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3a546-6465-468b-87fb-fcc5c803697f service nova] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:41.158407 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3a546-6465-468b-87fb-fcc5c803697f service nova] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:41.158705 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3a546-6465-468b-87fb-fcc5c803697f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:41.160579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-45d3a546-6465-468b-87fb-fcc5c803697f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:41.351321 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:41.398186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3a546-6465-468b-87fb-fcc5c803697f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:45:41.399859 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3a546-6465-468b-87fb-fcc5c803697f service nova] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:45:41.420167 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 710/1428] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:45:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:45:42.346851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd2547-5abb-4ca7-baef-a27b1de2badb None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:42.347479 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd2547-5abb-4ca7-baef-a27b1de2badb None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:42.347479 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd2547-5abb-4ca7-baef-a27b1de2badb None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:42.348701 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 719/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:45:42.354447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be4dc0-9e51-44df-8aa1-bfaa6a60cc6d None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:42.354990 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be4dc0-9e51-44df-8aa1-bfaa6a60cc6d None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:42.355095 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be4dc0-9e51-44df-8aa1-bfaa6a60cc6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:42.434114 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-05be4dc0-9e51-44df-8aa1-bfaa6a60cc6d None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:42.435344 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-05be4dc0-9e51-44df-8aa1-bfaa6a60cc6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:42.535364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 711/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:45:42.688694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd2547-5abb-4ca7-baef-a27b1de2badb None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:45:42.690388 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-76dd2547-5abb-4ca7-baef-a27b1de2badb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:42.745456 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:42.847578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd2547-5abb-4ca7-baef-a27b1de2badb None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:45:42.976082 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54778de-c19c-4b65-829b-c695c9b39d4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:45:42.976597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54778de-c19c-4b65-829b-c695c9b39d4b service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:42.976867 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54778de-c19c-4b65-829b-c695c9b39d4b service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:42.977132 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54778de-c19c-4b65-829b-c695c9b39d4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:42.978795 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f54778de-c19c-4b65-829b-c695c9b39d4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:42.985928 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:43.113331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54778de-c19c-4b65-829b-c695c9b39d4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:45:43.115953 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54778de-c19c-4b65-829b-c695c9b39d4b service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:45:43.139395 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 720/1431] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:45:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:45:44.036779 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545193e-5775-49ef-8737-dff46dedccda None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:44.037343 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545193e-5775-49ef-8737-dff46dedccda None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:44.037918 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545193e-5775-49ef-8737-dff46dedccda None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:44.039404 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 712/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:45:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:45:44.044984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df94066-908e-460e-a82b-8da09c335925 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:44.045544 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df94066-908e-460e-a82b-8da09c335925 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:44.045544 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df94066-908e-460e-a82b-8da09c335925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:44.131845 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-8df94066-908e-460e-a82b-8da09c335925 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:44.133090 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-8df94066-908e-460e-a82b-8da09c335925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:44.214660 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:44.254167 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 721/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:45:44.411210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545193e-5775-49ef-8737-dff46dedccda None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:45:44.413327 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-3545193e-5775-49ef-8737-dff46dedccda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:44.433458 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:44.579601 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545193e-5775-49ef-8737-dff46dedccda None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:45:44.674264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:44.702029 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a053329-f8bb-44ce-a749-1a6e6dc769a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:45:44.702698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a053329-f8bb-44ce-a749-1a6e6dc769a9 service nova] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:44.702985 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a053329-f8bb-44ce-a749-1a6e6dc769a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:44.703314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a053329-f8bb-44ce-a749-1a6e6dc769a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:44.705016 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-0a053329-f8bb-44ce-a749-1a6e6dc769a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:44.805172 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a053329-f8bb-44ce-a749-1a6e6dc769a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:45:44.806825 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a053329-f8bb-44ce-a749-1a6e6dc769a9 service nova] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:45:44.827475 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 713/1434] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:45:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:45:45.750397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0fb5e-d099-498d-9284-c0f0b11371e2 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:45.750988 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0fb5e-d099-498d-9284-c0f0b11371e2 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:45.750988 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0fb5e-d099-498d-9284-c0f0b11371e2 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:45.752350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 722/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:45:45.757954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3335de-b8f0-4a34-9444-d28a1031646e None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:45.758445 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3335de-b8f0-4a34-9444-d28a1031646e None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:45.758550 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3335de-b8f0-4a34-9444-d28a1031646e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:45.837310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-9c3335de-b8f0-4a34-9444-d28a1031646e None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:45.838504 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-9c3335de-b8f0-4a34-9444-d28a1031646e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:45.883233 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:45.948196 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 714/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:45:46.107529 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e0fb5e-d099-498d-9284-c0f0b11371e2 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:45:46.108950 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-05e0fb5e-d099-498d-9284-c0f0b11371e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:46.265924 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e0fb5e-d099-498d-9284-c0f0b11371e2 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:45:46.328792 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:46.399388 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f775a2-77bf-4ce6-9b06-c37d6eb2d0c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:45:46.400066 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f775a2-77bf-4ce6-9b06-c37d6eb2d0c0 service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:46.400385 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f775a2-77bf-4ce6-9b06-c37d6eb2d0c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:46.400701 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f775a2-77bf-4ce6-9b06-c37d6eb2d0c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:46.402543 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-87f775a2-77bf-4ce6-9b06-c37d6eb2d0c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:46.511058 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f775a2-77bf-4ce6-9b06-c37d6eb2d0c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:45:46.512683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f775a2-77bf-4ce6-9b06-c37d6eb2d0c0 service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:45:46.533125 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 723/1437] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:45:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:45:47.419635 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289bd60-f020-4f2f-abe3-cd42021f413d None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:47.420519 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289bd60-f020-4f2f-abe3-cd42021f413d None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:47.420876 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289bd60-f020-4f2f-abe3-cd42021f413d None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:47.422354 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 715/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:45:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:45:47.428286 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15851cff-35a5-4442-aa2a-205448bfdf6b None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:47.429313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15851cff-35a5-4442-aa2a-205448bfdf6b None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:47.429635 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15851cff-35a5-4442-aa2a-205448bfdf6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:47.510259 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-15851cff-35a5-4442-aa2a-205448bfdf6b None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:47.511518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-15851cff-35a5-4442-aa2a-205448bfdf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:47.513582 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:47.628718 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 724/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:45:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:45:47.786639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289bd60-f020-4f2f-abe3-cd42021f413d None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:45:47.789230 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-3289bd60-f020-4f2f-abe3-cd42021f413d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:47.954458 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289bd60-f020-4f2f-abe3-cd42021f413d None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:45:47.986156 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:48.087994 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a5aae-46bc-44b1-952c-78ae09c6c907 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:45:48.088509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a5aae-46bc-44b1-952c-78ae09c6c907 service nova] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:48.088772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a5aae-46bc-44b1-952c-78ae09c6c907 service nova] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:48.089034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a5aae-46bc-44b1-952c-78ae09c6c907 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:48.090696 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-602a5aae-46bc-44b1-952c-78ae09c6c907 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:48.197913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a5aae-46bc-44b1-952c-78ae09c6c907 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:45:48.199578 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a5aae-46bc-44b1-952c-78ae09c6c907 service nova] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:45:48.220057 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 716/1440] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:45:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:45:50.212002 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f04895-d091-4d54-a312-8ffeb5470855 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:50.212542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f04895-d091-4d54-a312-8ffeb5470855 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:50.212542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f04895-d091-4d54-a312-8ffeb5470855 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:50.214186 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 725/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:45:50.220165 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd36653-55bc-4fee-a20b-e4dd21737cf9 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:50.220923 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd36653-55bc-4fee-a20b-e4dd21737cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:50.220923 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd36653-55bc-4fee-a20b-e4dd21737cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:50.265845 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:50.293873 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-3dd36653-55bc-4fee-a20b-e4dd21737cf9 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:50.295142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-3dd36653-55bc-4fee-a20b-e4dd21737cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:50.398044 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 717/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:45:50.437187 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f04895-d091-4d54-a312-8ffeb5470855 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:45:50.439080 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-47f04895-d091-4d54-a312-8ffeb5470855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:50.553102 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f04895-d091-4d54-a312-8ffeb5470855 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:45:50.588803 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:50.632098 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903964fe-bdd7-4766-b30f-b94fc4f9ad77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:45:50.632725 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903964fe-bdd7-4766-b30f-b94fc4f9ad77 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:50.633000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903964fe-bdd7-4766-b30f-b94fc4f9ad77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:50.633266 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903964fe-bdd7-4766-b30f-b94fc4f9ad77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:50.634904 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-903964fe-bdd7-4766-b30f-b94fc4f9ad77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:50.760129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903964fe-bdd7-4766-b30f-b94fc4f9ad77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:45:50.761701 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903964fe-bdd7-4766-b30f-b94fc4f9ad77 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:45:50.784460 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 726/1443] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:45:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:45:53.039733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd34d0-f2a5-49c3-a8d1-63245a5c6451 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:53.040325 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd34d0-f2a5-49c3-a8d1-63245a5c6451 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:53.040325 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd34d0-f2a5-49c3-a8d1-63245a5c6451 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:53.041697 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 718/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:45:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:45:53.048685 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc153606-199a-4692-9842-9ac095a189f2 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:53.049259 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc153606-199a-4692-9842-9ac095a189f2 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:53.049607 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc153606-199a-4692-9842-9ac095a189f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:53.135225 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-dc153606-199a-4692-9842-9ac095a189f2 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:53.136469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-dc153606-199a-4692-9842-9ac095a189f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:53.151474 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:53.249337 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 727/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:45:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:45:53.291606 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd34d0-f2a5-49c3-a8d1-63245a5c6451 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:45:53.293325 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-73dd34d0-f2a5-49c3-a8d1-63245a5c6451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:53.413884 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd34d0-f2a5-49c3-a8d1-63245a5c6451 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:45:53.427232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:53.501927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac747f-c948-400b-8598-9cbde4c2deae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:45:53.502443 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac747f-c948-400b-8598-9cbde4c2deae service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:53.502752 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac747f-c948-400b-8598-9cbde4c2deae service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:53.503021 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac747f-c948-400b-8598-9cbde4c2deae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:53.504660 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-7bac747f-c948-400b-8598-9cbde4c2deae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:53.626710 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac747f-c948-400b-8598-9cbde4c2deae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:45:53.628148 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac747f-c948-400b-8598-9cbde4c2deae service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:45:53.648410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 719/1446] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:45:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:45:56.392805 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81770ed-05bc-40b9-93dc-fefc5590bd7d None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:56.393445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81770ed-05bc-40b9-93dc-fefc5590bd7d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:56.393445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81770ed-05bc-40b9-93dc-fefc5590bd7d None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:56.394714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 728/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:45:56.401026 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c48c9-aab9-4a52-a697-d8809ffa2b60 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:56.401638 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c48c9-aab9-4a52-a697-d8809ffa2b60 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:56.401638 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c48c9-aab9-4a52-a697-d8809ffa2b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:56.473696 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:56.497519 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-582c48c9-aab9-4a52-a697-d8809ffa2b60 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:56.498849 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-582c48c9-aab9-4a52-a697-d8809ffa2b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:56.609508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 720/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:45:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:45:56.652684 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81770ed-05bc-40b9-93dc-fefc5590bd7d None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:45:56.654376 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b81770ed-05bc-40b9-93dc-fefc5590bd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:56.776318 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81770ed-05bc-40b9-93dc-fefc5590bd7d None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:45:56.827658 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:56.869238 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d2ca3-d877-47c6-8c5a-4afdd810ffea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:45:56.869864 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d2ca3-d877-47c6-8c5a-4afdd810ffea service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:56.870138 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d2ca3-d877-47c6-8c5a-4afdd810ffea service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:56.870418 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d2ca3-d877-47c6-8c5a-4afdd810ffea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:56.872237 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-767d2ca3-d877-47c6-8c5a-4afdd810ffea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:57.072543 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d2ca3-d877-47c6-8c5a-4afdd810ffea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:45:57.075080 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d2ca3-d877-47c6-8c5a-4afdd810ffea service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:45:57.109632 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 729/1449] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:45:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:45:59.383715 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16314bb-d8c7-4e1c-88cf-485f25d19e15 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:59.384382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16314bb-d8c7-4e1c-88cf-485f25d19e15 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:59.384382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16314bb-d8c7-4e1c-88cf-485f25d19e15 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:59.385596 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 721/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:45:59.391473 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce63b1-e5a2-41ed-9c0a-98419291578f None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:59.392457 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce63b1-e5a2-41ed-9c0a-98419291578f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:59.392457 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce63b1-e5a2-41ed-9c0a-98419291578f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:59.473090 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-a7ce63b1-e5a2-41ed-9c0a-98419291578f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:45:59.475671 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-a7ce63b1-e5a2-41ed-9c0a-98419291578f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:59.499234 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:59.585061 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 730/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:45:59.626362 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16314bb-d8c7-4e1c-88cf-485f25d19e15 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:45:59.628109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-f16314bb-d8c7-4e1c-88cf-485f25d19e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:59.780718 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16314bb-d8c7-4e1c-88cf-485f25d19e15 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:45:59.802971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:45:59.861559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b3785-c785-4f26-a287-9f4b98be29a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:45:59.862164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b3785-c785-4f26-a287-9f4b98be29a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:45:59.862434 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b3785-c785-4f26-a287-9f4b98be29a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:45:59.862749 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b3785-c785-4f26-a287-9f4b98be29a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:45:59.864328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-9d5b3785-c785-4f26-a287-9f4b98be29a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:45:59.989431 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b3785-c785-4f26-a287-9f4b98be29a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:45:59.991536 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b3785-c785-4f26-a287-9f4b98be29a6 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:46:00.011956 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 722/1452] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:45:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:46:02.200278 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0383df-1b9e-407c-b8af-65ebb004c49d None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:02.200862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0383df-1b9e-407c-b8af-65ebb004c49d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:02.200862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0383df-1b9e-407c-b8af-65ebb004c49d None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:02.202272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 731/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:46:02.209087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982e11a-c687-43cf-b9cb-d2964e5778b6 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:02.209880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982e11a-c687-43cf-b9cb-d2964e5778b6 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:02.210053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982e11a-c687-43cf-b9cb-d2964e5778b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:02.282135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:02.285382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-f982e11a-c687-43cf-b9cb-d2964e5778b6 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:46:02.286858 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-f982e11a-c687-43cf-b9cb-d2964e5778b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:02.401649 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 723/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:46:02.441909 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0383df-1b9e-407c-b8af-65ebb004c49d None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:46:02.444122 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-fc0383df-1b9e-407c-b8af-65ebb004c49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:02.560228 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0383df-1b9e-407c-b8af-65ebb004c49d None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:46:02.619469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:02.647896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04228f87-2651-4cce-b756-ebd8b37a287f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:46:02.648540 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04228f87-2651-4cce-b756-ebd8b37a287f service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:02.648796 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04228f87-2651-4cce-b756-ebd8b37a287f service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:02.649318 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04228f87-2651-4cce-b756-ebd8b37a287f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:02.651737 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-04228f87-2651-4cce-b756-ebd8b37a287f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:02.778587 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04228f87-2651-4cce-b756-ebd8b37a287f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:46:02.780051 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04228f87-2651-4cce-b756-ebd8b37a287f service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:46:02.800948 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 732/1455] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:46:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:46:04.970215 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7ed89-f83b-4b93-9bad-f9b632d32107 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:04.970881 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7ed89-f83b-4b93-9bad-f9b632d32107 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:04.970881 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7ed89-f83b-4b93-9bad-f9b632d32107 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:04.972141 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 724/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:46:04.978769 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b0414-7f70-4482-a937-5345fcab9ecc None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:04.979308 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b0414-7f70-4482-a937-5345fcab9ecc None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:04.979308 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b0414-7f70-4482-a937-5345fcab9ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:05.051163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-047b0414-7f70-4482-a937-5345fcab9ecc None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:46:05.052608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-047b0414-7f70-4482-a937-5345fcab9ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:05.084527 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:05.158490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 733/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:46:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:46:05.200486 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7ed89-f83b-4b93-9bad-f9b632d32107 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:46:05.202018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-6ac7ed89-f83b-4b93-9bad-f9b632d32107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:05.353949 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7ed89-f83b-4b93-9bad-f9b632d32107 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:46:05.404873 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:05.455863 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7194e6-75a3-4b7a-95db-a4b6f3c16593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:46:05.456418 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7194e6-75a3-4b7a-95db-a4b6f3c16593 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:05.456698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7194e6-75a3-4b7a-95db-a4b6f3c16593 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:05.456969 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7194e6-75a3-4b7a-95db-a4b6f3c16593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:05.459584 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-be7194e6-75a3-4b7a-95db-a4b6f3c16593 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:05.585081 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7194e6-75a3-4b7a-95db-a4b6f3c16593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:46:05.586544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7194e6-75a3-4b7a-95db-a4b6f3c16593 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:46:05.606976 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 725/1458] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:46:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:46:07.845392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab955a-8513-462a-8657-e409da093484 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:07.845971 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab955a-8513-462a-8657-e409da093484 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:07.845971 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab955a-8513-462a-8657-e409da093484 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:07.847311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 734/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:46:07.852992 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5c932-2f38-428b-a9a2-dd78bd81d33e None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:07.853612 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5c932-2f38-428b-a9a2-dd78bd81d33e None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:07.853612 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5c932-2f38-428b-a9a2-dd78bd81d33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:07.922939 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-c1d5c932-2f38-428b-a9a2-dd78bd81d33e None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:46:07.924223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-c1d5c932-2f38-428b-a9a2-dd78bd81d33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:07.933865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:08.047500 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 726/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:46:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:46:08.088497 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab955a-8513-462a-8657-e409da093484 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:46:08.090516 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-32ab955a-8513-462a-8657-e409da093484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:08.206233 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab955a-8513-462a-8657-e409da093484 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:46:08.269705 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:08.291225 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2397b2-16cb-4f76-8640-766b85e7500a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:46:08.292135 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2397b2-16cb-4f76-8640-766b85e7500a service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:08.292411 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2397b2-16cb-4f76-8640-766b85e7500a service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:08.292674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2397b2-16cb-4f76-8640-766b85e7500a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:08.294411 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-7c2397b2-16cb-4f76-8640-766b85e7500a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:08.422497 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2397b2-16cb-4f76-8640-766b85e7500a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:46:08.423999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2397b2-16cb-4f76-8640-766b85e7500a service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:46:08.445404 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 735/1461] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:46:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:46:10.604025 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eba1c3-b182-4781-9942-2cec6cf85b32 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:10.604626 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eba1c3-b182-4781-9942-2cec6cf85b32 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:10.604759 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eba1c3-b182-4781-9942-2cec6cf85b32 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:10.606366 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 727/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:46:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:46:10.613738 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad548d5-5c5d-412f-8016-78ed2b300b22 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:10.614445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad548d5-5c5d-412f-8016-78ed2b300b22 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:10.614764 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad548d5-5c5d-412f-8016-78ed2b300b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:10.704416 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-0ad548d5-5c5d-412f-8016-78ed2b300b22 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:46:10.705923 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-0ad548d5-5c5d-412f-8016-78ed2b300b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:10.754222 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:10.829113 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 736/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:46:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:46:10.868040 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eba1c3-b182-4781-9942-2cec6cf85b32 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:46:10.869611 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-a4eba1c3-b182-4781-9942-2cec6cf85b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:11.000744 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eba1c3-b182-4781-9942-2cec6cf85b32 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:46:11.043260 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:11.082261 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d3cfa-023c-4aec-b768-d1881e796eb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:46:11.082895 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d3cfa-023c-4aec-b768-d1881e796eb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:11.083201 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d3cfa-023c-4aec-b768-d1881e796eb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:11.084063 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d3cfa-023c-4aec-b768-d1881e796eb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:11.085669 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-0f5d3cfa-023c-4aec-b768-d1881e796eb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:11.203562 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d3cfa-023c-4aec-b768-d1881e796eb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:46:11.204930 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d3cfa-023c-4aec-b768-d1881e796eb8 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:46:11.235608 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 728/1464] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:46:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:46:13.398262 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11713b2d-aefa-4b28-8151-6d99f8dabb7a None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:13.398919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11713b2d-aefa-4b28-8151-6d99f8dabb7a None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:13.398919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11713b2d-aefa-4b28-8151-6d99f8dabb7a None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:13.400182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 737/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:46:13.405926 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554abdc-c784-41c4-b153-9912892d1202 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:13.406411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554abdc-c784-41c4-b153-9912892d1202 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:13.406533 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554abdc-c784-41c4-b153-9912892d1202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:13.477150 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-3554abdc-c784-41c4-b153-9912892d1202 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:46:13.478363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-3554abdc-c784-41c4-b153-9912892d1202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:13.498070 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:13.581142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 729/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:46:13.618188 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11713b2d-aefa-4b28-8151-6d99f8dabb7a None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:46:13.619776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-11713b2d-aefa-4b28-8151-6d99f8dabb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:13.735021 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11713b2d-aefa-4b28-8151-6d99f8dabb7a None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:46:13.804723 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:13.811667 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5c740-cf1d-4f85-8da7-25abb980cbd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:46:13.812277 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5c740-cf1d-4f85-8da7-25abb980cbd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:13.812560 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5c740-cf1d-4f85-8da7-25abb980cbd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:13.812827 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5c740-cf1d-4f85-8da7-25abb980cbd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:13.814456 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-65f5c740-cf1d-4f85-8da7-25abb980cbd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:13.933360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5c740-cf1d-4f85-8da7-25abb980cbd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:46:13.935177 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5c740-cf1d-4f85-8da7-25abb980cbd3 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:46:13.971074 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 738/1467] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:46:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:46:16.047591 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d035c6-cc02-4c25-ac2a-8ce7cb76c3c6 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:16.048136 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d035c6-cc02-4c25-ac2a-8ce7cb76c3c6 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:16.048136 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d035c6-cc02-4c25-ac2a-8ce7cb76c3c6 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:16.049619 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 730/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:46:16.056249 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17f8fc-8c19-42a8-a089-9d99ad58ec6f None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:16.056788 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17f8fc-8c19-42a8-a089-9d99ad58ec6f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:16.056788 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17f8fc-8c19-42a8-a089-9d99ad58ec6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:16.131113 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-bc17f8fc-8c19-42a8-a089-9d99ad58ec6f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:46:16.132412 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-bc17f8fc-8c19-42a8-a089-9d99ad58ec6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:16.204824 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:16.264791 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 739/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:46:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:46:16.303460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d035c6-cc02-4c25-ac2a-8ce7cb76c3c6 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:46:16.305180 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-18d035c6-cc02-4c25-ac2a-8ce7cb76c3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:16.420198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d035c6-cc02-4c25-ac2a-8ce7cb76c3c6 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:46:16.472954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:16.501804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8391bc3-aa0b-489c-9c1b-b5f0b7b38348 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:46:16.502423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8391bc3-aa0b-489c-9c1b-b5f0b7b38348 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:16.502985 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8391bc3-aa0b-489c-9c1b-b5f0b7b38348 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:16.503410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8391bc3-aa0b-489c-9c1b-b5f0b7b38348 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:16.505402 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-c8391bc3-aa0b-489c-9c1b-b5f0b7b38348 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:16.623278 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8391bc3-aa0b-489c-9c1b-b5f0b7b38348 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:46:16.624719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8391bc3-aa0b-489c-9c1b-b5f0b7b38348 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:46:16.646957 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 731/1470] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:46:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:46:26.262922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=22 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:26.642979 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=66 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:37.049397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03012e-0d94-4820-ace3-eb0d0a6f015a None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:37.050006 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03012e-0d94-4820-ace3-eb0d0a6f015a None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:37.050006 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03012e-0d94-4820-ace3-eb0d0a6f015a None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:37.051582 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 740/1471] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:46:37.059399 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab909452-df34-44d7-8d2d-a45de76ccd88 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:37.059870 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab909452-df34-44d7-8d2d-a45de76ccd88 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:37.060071 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab909452-df34-44d7-8d2d-a45de76ccd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:37.133059 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-ab909452-df34-44d7-8d2d-a45de76ccd88 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:46:37.134287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ab909452-df34-44d7-8d2d-a45de76ccd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:37.238881 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 732/1472] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:46:37.330486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5e271-c6f6-4ca6-bfc6-54bae6190536 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:37.331070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5e271-c6f6-4ca6-bfc6-54bae6190536 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:37.331369 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5e271-c6f6-4ca6-bfc6-54bae6190536 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:37.333861 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 741/1473] 10.222.0.13 () {54 vars in 967 bytes} [Mon Aug 8 16:46:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:46:37.339921 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ebe6b-7a24-4642-ac86-a7341b580259 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:37.340382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ebe6b-7a24-4642-ac86-a7341b580259 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:37.340834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ebe6b-7a24-4642-ac86-a7341b580259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:37.429931 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-af2ebe6b-7a24-4642-ac86-a7341b580259 None None] MFA Rules not processed for user `ea51ef02128a4b63b11049eb0daef298`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:46:37.431255 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-af2ebe6b-7a24-4642-ac86-a7341b580259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:37.459267 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:37.518652 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 733/1474] 10.222.0.13 () {58 vars in 1064 bytes} [Mon Aug 8 16:46:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:46:37.526882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5e271-c6f6-4ca6-bfc6-54bae6190536 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:46:37.528466 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-dbc5e271-c6f6-4ca6-bfc6-54bae6190536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:37.638873 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5e271-c6f6-4ca6-bfc6-54bae6190536 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:46:37.720550 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd36a27-27ea-4a95-b722-3b10097946b7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea51ef02128a4b63b11049eb0daef298', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:46:37.721163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd36a27-27ea-4a95-b722-3b10097946b7 service glance] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:37.721420 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd36a27-27ea-4a95-b722-3b10097946b7 service glance] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:37.721691 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd36a27-27ea-4a95-b722-3b10097946b7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:37.723454 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-1dd36a27-27ea-4a95-b722-3b10097946b7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:37.786216 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:37.845927 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd36a27-27ea-4a95-b722-3b10097946b7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:46:37.847487 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd36a27-27ea-4a95-b722-3b10097946b7 service glance] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:46:37.868580 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 742/1475] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:46:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:46:40.049437 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64a320-5b28-484a-b042-e55e183fdd81 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:40.050170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64a320-5b28-484a-b042-e55e183fdd81 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:40.050170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64a320-5b28-484a-b042-e55e183fdd81 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:40.051979 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 734/1476] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:46:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:46:40.057884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c74fd-0cbc-470f-b0c2-ff45de183463 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:40.058417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c74fd-0cbc-470f-b0c2-ff45de183463 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:40.058417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c74fd-0cbc-470f-b0c2-ff45de183463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:40.125719 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-777c74fd-0cbc-470f-b0c2-ff45de183463 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:46:40.126968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-777c74fd-0cbc-470f-b0c2-ff45de183463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:40.227339 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 743/1477] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:46:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:46:40.250414 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64a320-5b28-484a-b042-e55e183fdd81 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:46:40.252003 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-7e64a320-5b28-484a-b042-e55e183fdd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:40.379861 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64a320-5b28-484a-b042-e55e183fdd81 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:46:40.429413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:40.476735 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e0176-dcdd-4ece-b25d-c1ceb67ac4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:46:40.477399 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e0176-dcdd-4ece-b25d-c1ceb67ac4eb admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:40.477651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e0176-dcdd-4ece-b25d-c1ceb67ac4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:40.477917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e0176-dcdd-4ece-b25d-c1ceb67ac4eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:40.478557 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e0176-dcdd-4ece-b25d-c1ceb67ac4eb admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:46:40.480392 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: WARNING py.warnings [None req-948e0176-dcdd-4ece-b25d-c1ceb67ac4eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:46:40.480392 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: warnings.warn(msg) Aug 08 16:46:40.480392 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: Aug 08 16:46:40.481522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e0176-dcdd-4ece-b25d-c1ceb67ac4eb admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:46:40.501521 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 735/1478] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Aug 8 16:46:40 2022] GET /identity/v3/projects => generated 2553 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:46:42.595323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31039a3b-dca4-4dc7-82c2-4e993c002174 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:42.596021 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31039a3b-dca4-4dc7-82c2-4e993c002174 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:42.596021 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31039a3b-dca4-4dc7-82c2-4e993c002174 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:42.597293 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 744/1479] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:46:42.603801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7d0b6-a0f2-4adb-8d82-50dfab288b4d None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:42.604315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7d0b6-a0f2-4adb-8d82-50dfab288b4d None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:42.604315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7d0b6-a0f2-4adb-8d82-50dfab288b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:42.681756 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-9fd7d0b6-a0f2-4adb-8d82-50dfab288b4d None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:46:42.682999 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-9fd7d0b6-a0f2-4adb-8d82-50dfab288b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:42.793089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 736/1480] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:46:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:46:42.840043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31039a3b-dca4-4dc7-82c2-4e993c002174 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:46:42.841607 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-31039a3b-dca4-4dc7-82c2-4e993c002174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:42.853269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:43.004934 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31039a3b-dca4-4dc7-82c2-4e993c002174 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:46:43.086294 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:43.119878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b6231-919e-41b7-9801-f0253b89d25a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:46:43.120486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b6231-919e-41b7-9801-f0253b89d25a service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:43.120754 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b6231-919e-41b7-9801-f0253b89d25a service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:43.121018 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b6231-919e-41b7-9801-f0253b89d25a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:43.122651 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-523b6231-919e-41b7-9801-f0253b89d25a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:43.216176 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523b6231-919e-41b7-9801-f0253b89d25a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:46:43.217765 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523b6231-919e-41b7-9801-f0253b89d25a service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:46:43.236428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 745/1481] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:46:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:46:45.462333 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40cbc41-3c9a-4c3f-a84d-22cb7ff312f0 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:45.462957 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40cbc41-3c9a-4c3f-a84d-22cb7ff312f0 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:45.462957 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40cbc41-3c9a-4c3f-a84d-22cb7ff312f0 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:45.464280 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 737/1482] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:46:45.469728 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68dce84-80c8-4f95-9194-56a6290c3e83 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:45.470296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68dce84-80c8-4f95-9194-56a6290c3e83 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:45.470296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68dce84-80c8-4f95-9194-56a6290c3e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:45.542890 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-f68dce84-80c8-4f95-9194-56a6290c3e83 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:46:45.544144 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f68dce84-80c8-4f95-9194-56a6290c3e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:45.598335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:45.656629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 746/1483] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:46:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:46:45.694681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40cbc41-3c9a-4c3f-a84d-22cb7ff312f0 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:46:45.696210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d40cbc41-3c9a-4c3f-a84d-22cb7ff312f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:45.721339 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:45.804492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40cbc41-3c9a-4c3f-a84d-22cb7ff312f0 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:46:45.876310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd858e-cb81-42a2-9b57-8460ecd5e83f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:46:45.876905 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd858e-cb81-42a2-9b57-8460ecd5e83f service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:45.877175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd858e-cb81-42a2-9b57-8460ecd5e83f service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:45.877436 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd858e-cb81-42a2-9b57-8460ecd5e83f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:45.879062 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-66fd858e-cb81-42a2-9b57-8460ecd5e83f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:45.956620 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:45.994700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fd858e-cb81-42a2-9b57-8460ecd5e83f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:46:45.996071 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fd858e-cb81-42a2-9b57-8460ecd5e83f service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:46:46.016029 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 738/1484] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:46:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:46:48.141507 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b816f-a9a7-41ba-88e1-c58d82968d11 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:48.142139 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b816f-a9a7-41ba-88e1-c58d82968d11 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:48.142139 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b816f-a9a7-41ba-88e1-c58d82968d11 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:48.143447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 747/1485] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:46:48.149983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e6a11-dfca-4214-949f-28d2708b0745 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:48.150456 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e6a11-dfca-4214-949f-28d2708b0745 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:48.150456 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e6a11-dfca-4214-949f-28d2708b0745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:48.222001 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-168e6a11-dfca-4214-949f-28d2708b0745 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:46:48.224084 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-168e6a11-dfca-4214-949f-28d2708b0745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:48.328178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 739/1486] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:46:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:46:48.368139 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b816f-a9a7-41ba-88e1-c58d82968d11 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:46:48.369891 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-5c3b816f-a9a7-41ba-88e1-c58d82968d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:48.483125 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b816f-a9a7-41ba-88e1-c58d82968d11 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:46:48.558484 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:48.560337 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90662a-d70d-4067-ac64-69bd1f270d12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:46:48.560977 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90662a-d70d-4067-ac64-69bd1f270d12 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:48.561257 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90662a-d70d-4067-ac64-69bd1f270d12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:48.561560 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90662a-d70d-4067-ac64-69bd1f270d12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:48.563229 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ed90662a-d70d-4067-ac64-69bd1f270d12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:48.679830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed90662a-d70d-4067-ac64-69bd1f270d12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:46:48.681271 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed90662a-d70d-4067-ac64-69bd1f270d12 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:46:48.700307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 748/1487] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:46:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:46:50.892119 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b4e8b-b6af-452f-a8a3-fb5f8298b3ee None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:50.892699 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b4e8b-b6af-452f-a8a3-fb5f8298b3ee None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:50.892699 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b4e8b-b6af-452f-a8a3-fb5f8298b3ee None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:50.894041 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 740/1488] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:46:50.899548 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be5e2b-6161-4813-8ac1-118284942db6 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:50.900024 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be5e2b-6161-4813-8ac1-118284942db6 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:50.900130 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be5e2b-6161-4813-8ac1-118284942db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:50.970466 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-c0be5e2b-6161-4813-8ac1-118284942db6 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:46:50.971727 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-c0be5e2b-6161-4813-8ac1-118284942db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:51.026297 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:51.071572 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 749/1489] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:46:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:46:51.113064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b4e8b-b6af-452f-a8a3-fb5f8298b3ee None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:46:51.115473 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-949b4e8b-b6af-452f-a8a3-fb5f8298b3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:51.152748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:51.224311 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b4e8b-b6af-452f-a8a3-fb5f8298b3ee None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:46:51.298893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84306652-74ca-4c1c-b06c-203f49b85a8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:46:51.299504 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84306652-74ca-4c1c-b06c-203f49b85a8d service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:51.299799 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84306652-74ca-4c1c-b06c-203f49b85a8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:51.300063 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84306652-74ca-4c1c-b06c-203f49b85a8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:51.302067 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-84306652-74ca-4c1c-b06c-203f49b85a8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:51.386422 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:51.418175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84306652-74ca-4c1c-b06c-203f49b85a8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:46:51.419613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84306652-74ca-4c1c-b06c-203f49b85a8d service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:46:51.439833 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 741/1490] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:46:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:46:53.916622 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622570be-23b1-43f6-be43-a15268bd6b9a None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:53.917225 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622570be-23b1-43f6-be43-a15268bd6b9a None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:53.917225 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622570be-23b1-43f6-be43-a15268bd6b9a None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:53.918512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 750/1491] 10.222.0.13 () {54 vars in 927 bytes} [Mon Aug 8 16:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:46:53.923941 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe04b6-5b3a-4c09-bd27-d7a361cc8c44 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:53.924418 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe04b6-5b3a-4c09-bd27-d7a361cc8c44 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:53.924513 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe04b6-5b3a-4c09-bd27-d7a361cc8c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:53.992032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-50fe04b6-5b3a-4c09-bd27-d7a361cc8c44 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:46:53.993236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-50fe04b6-5b3a-4c09-bd27-d7a361cc8c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:54.090233 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 742/1492] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Aug 8 16:46:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:46:54.130294 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622570be-23b1-43f6-be43-a15268bd6b9a None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:46:54.131927 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-622570be-23b1-43f6-be43-a15268bd6b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:54.240400 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622570be-23b1-43f6-be43-a15268bd6b9a None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:46:54.315918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1caf5-46b9-41cf-adbb-7096bdc23f32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:46:54.317294 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1caf5-46b9-41cf-adbb-7096bdc23f32 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:46:54.317788 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1caf5-46b9-41cf-adbb-7096bdc23f32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:46:54.318183 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1caf5-46b9-41cf-adbb-7096bdc23f32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:46:54.319850 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-00c1caf5-46b9-41cf-adbb-7096bdc23f32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:46:54.329218 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:46:54.432720 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c1caf5-46b9-41cf-adbb-7096bdc23f32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:46:54.434174 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c1caf5-46b9-41cf-adbb-7096bdc23f32 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:46:54.453955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 751/1493] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:46:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:47:04.088549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=82 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:47:04.451562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=50 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:48:15.387786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c6627-0365-49d0-ba3e-d09ddc9680e8 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:48:15.388471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c6627-0365-49d0-ba3e-d09ddc9680e8 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:48:15.388471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c6627-0365-49d0-ba3e-d09ddc9680e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:48:15.467783 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-047c6627-0365-49d0-ba3e-d09ddc9680e8 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:48:15.469008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-047c6627-0365-49d0-ba3e-d09ddc9680e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:48:15.568063 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 743/1494] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:48:15.576373 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c3440-a511-498c-9f13-426517ac5d70 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:48:15.576894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c3440-a511-498c-9f13-426517ac5d70 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:48:15.576894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c3440-a511-498c-9f13-426517ac5d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:48:15.653975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-6b1c3440-a511-498c-9f13-426517ac5d70 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:48:15.655122 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-6b1c3440-a511-498c-9f13-426517ac5d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:48:15.755437 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 752/1495] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:48:15.760271 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c6627-0365-49d0-ba3e-d09ddc9680e8 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:48:15.761468 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-047c6627-0365-49d0-ba3e-d09ddc9680e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:48:15.773049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c6627-0365-49d0-ba3e-d09ddc9680e8 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:48:15.836573 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:48:15.871181 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db4992-2eb8-4ee3-b888-fdfb535e81e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:48:15.871796 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db4992-2eb8-4ee3-b888-fdfb535e81e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:48:15.872075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db4992-2eb8-4ee3-b888-fdfb535e81e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:48:15.872347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db4992-2eb8-4ee3-b888-fdfb535e81e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:48:15.872987 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db4992-2eb8-4ee3-b888-fdfb535e81e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:48:15.873284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db4992-2eb8-4ee3-b888-fdfb535e81e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:48:15.874312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: WARNING py.warnings [None req-40db4992-2eb8-4ee3-b888-fdfb535e81e0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:48:15.874312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: warnings.warn(msg) Aug 08 16:48:15.874312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: Aug 08 16:48:15.875250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db4992-2eb8-4ee3-b888-fdfb535e81e0 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:48:15.886941 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 744/1496] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:48:15.896262 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c3440-a511-498c-9f13-426517ac5d70 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:48:15.905534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c3440-a511-498c-9f13-426517ac5d70 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:48:15.969143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:48:16.007065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed7da38-596c-449c-b977-02652954f420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:48:16.007678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7da38-596c-449c-b977-02652954f420 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:48:16.007954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7da38-596c-449c-b977-02652954f420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:48:16.008219 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7da38-596c-449c-b977-02652954f420 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:48:16.008843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed7da38-596c-449c-b977-02652954f420 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:48:16.010614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: WARNING py.warnings [None req-3ed7da38-596c-449c-b977-02652954f420 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:48:16.010614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: warnings.warn(msg) Aug 08 16:48:16.010614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: Aug 08 16:48:16.011805 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed7da38-596c-449c-b977-02652954f420 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:48:16.149448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 753/1497] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:48:15 2022] POST /identity/v3/projects => generated 377 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:48:16.153915 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db4992-2eb8-4ee3-b888-fdfb535e81e0 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:48:16.155937 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-40db4992-2eb8-4ee3-b888-fdfb535e81e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:48:16.191760 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db4992-2eb8-4ee3-b888-fdfb535e81e0 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:48:16.289704 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d99af-3846-4ebf-b7b8-eedda4245a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:48:16.290309 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d99af-3846-4ebf-b7b8-eedda4245a1e admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:48:16.290608 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d99af-3846-4ebf-b7b8-eedda4245a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:48:16.290884 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d99af-3846-4ebf-b7b8-eedda4245a1e admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:48:16.291504 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d99af-3846-4ebf-b7b8-eedda4245a1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:48:16.292538 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: WARNING py.warnings [None req-f89d99af-3846-4ebf-b7b8-eedda4245a1e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:48:16.292538 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: warnings.warn(msg) Aug 08 16:48:16.292538 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: Aug 08 16:48:16.293863 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d99af-3846-4ebf-b7b8-eedda4245a1e admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:48:16.464365 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 745/1498] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:48:16 2022] POST /identity/v3/users => generated 289 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:48:16.468720 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed7da38-596c-449c-b977-02652954f420 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:48:16.470243 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-3ed7da38-596c-449c-b977-02652954f420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:48:16.536655 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed7da38-596c-449c-b977-02652954f420 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:48:16.622472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:48:16.633362 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10063d77-6de9-4d52-8ff8-b2c64497ec9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:48:16.641054 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10063d77-6de9-4d52-8ff8-b2c64497ec9b admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:48:16.641054 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10063d77-6de9-4d52-8ff8-b2c64497ec9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:48:16.641054 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10063d77-6de9-4d52-8ff8-b2c64497ec9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:48:16.641054 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10063d77-6de9-4d52-8ff8-b2c64497ec9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:48:16.641054 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: WARNING py.warnings [None req-10063d77-6de9-4d52-8ff8-b2c64497ec9b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:48:16.641054 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: warnings.warn(msg) Aug 08 16:48:16.641054 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: Aug 08 16:48:16.641675 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10063d77-6de9-4d52-8ff8-b2c64497ec9b admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:48:16.650045 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 754/1499] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:48:16.654401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d99af-3846-4ebf-b7b8-eedda4245a1e admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:48:16.655975 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-f89d99af-3846-4ebf-b7b8-eedda4245a1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:48:16.669276 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d99af-3846-4ebf-b7b8-eedda4245a1e admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:48:16.697138 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:48:16.771663 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295227d-0e82-464f-879c-fe904fdf6e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:48:16.772195 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295227d-0e82-464f-879c-fe904fdf6e31 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:48:16.772469 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295227d-0e82-464f-879c-fe904fdf6e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:48:16.772734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295227d-0e82-464f-879c-fe904fdf6e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:48:16.773370 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f295227d-0e82-464f-879c-fe904fdf6e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:48:16.774390 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: WARNING py.warnings [None req-f295227d-0e82-464f-879c-fe904fdf6e31 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:48:16.774390 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: warnings.warn(msg) Aug 08 16:48:16.774390 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: Aug 08 16:48:16.775347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f295227d-0e82-464f-879c-fe904fdf6e31 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:48:16.787974 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 746/1500] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:48:16.792544 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10063d77-6de9-4d52-8ff8-b2c64497ec9b admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:48:16.794173 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-10063d77-6de9-4d52-8ff8-b2c64497ec9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:48:16.828551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10063d77-6de9-4d52-8ff8-b2c64497ec9b admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:48:16.931923 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae1734-8f4c-422a-9dfd-692f1bdf8d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:48:16.932597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae1734-8f4c-422a-9dfd-692f1bdf8d6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:48:16.932868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae1734-8f4c-422a-9dfd-692f1bdf8d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:48:16.933131 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae1734-8f4c-422a-9dfd-692f1bdf8d6a admin admin] PATH_INFO: `/v3/projects/b9f65f2e0903458aafe194fbd5c83b9c/users/7ec6488f8610445989cb432850839a54/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:48:16.959344 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae1734-8f4c-422a-9dfd-692f1bdf8d6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9f65f2e0903458aafe194fbd5c83b9c, user_id=7ec6488f8610445989cb432850839a54, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:48:16.960388 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: WARNING py.warnings [None req-7dae1734-8f4c-422a-9dfd-692f1bdf8d6a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:48:16.960388 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: warnings.warn(msg) Aug 08 16:48:16.960388 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: Aug 08 16:48:16.962351 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae1734-8f4c-422a-9dfd-692f1bdf8d6a admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:48:17.039017 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 755/1501] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:48:16 2022] PUT /identity/v3/projects/b9f65f2e0903458aafe194fbd5c83b9c/users/7ec6488f8610445989cb432850839a54/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:48:17.052200 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff43e1-7376-4e7b-9513-e2c89d913f77 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:48:17.052483 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff43e1-7376-4e7b-9513-e2c89d913f77 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:48:17.052748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff43e1-7376-4e7b-9513-e2c89d913f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:48:17.112295 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-81ff43e1-7376-4e7b-9513-e2c89d913f77 None None] MFA Rules not processed for user `7ec6488f8610445989cb432850839a54`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:48:17.113534 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-81ff43e1-7376-4e7b-9513-e2c89d913f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:48:17.195215 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 747/1502] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:48:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:48:17.209287 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a225c-5d1a-4c39-9ff9-9633b8b4dece None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:48:17.209572 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a225c-5d1a-4c39-9ff9-9633b8b4dece None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:48:17.209844 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a225c-5d1a-4c39-9ff9-9633b8b4dece None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:48:17.211225 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 756/1503] 10.222.0.13 () {54 vars in 964 bytes} [Mon Aug 8 16:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:48:17.216250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece30637-eef8-45a2-b0d4-42e089544ee1 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:48:17.216582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece30637-eef8-45a2-b0d4-42e089544ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:48:17.216895 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece30637-eef8-45a2-b0d4-42e089544ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:48:17.250119 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:48:17.296673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-ece30637-eef8-45a2-b0d4-42e089544ee1 None None] MFA Rules not processed for user `7bbf0678f34e46e8b56aaeabce955180`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:48:17.297799 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ece30637-eef8-45a2-b0d4-42e089544ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:48:17.374838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 748/1504] 10.222.0.13 () {58 vars in 1061 bytes} [Mon Aug 8 16:48:17 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:48:17.382900 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a225c-5d1a-4c39-9ff9-9633b8b4dece None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:48:17.384452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-727a225c-5d1a-4c39-9ff9-9633b8b4dece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:48:17.435544 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:48:17.488887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a225c-5d1a-4c39-9ff9-9633b8b4dece None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:48:17.564613 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d1b32-a1f8-4373-b920-564e2a436054 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:48:17.565201 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d1b32-a1f8-4373-b920-564e2a436054 service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:48:17.565504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d1b32-a1f8-4373-b920-564e2a436054 service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:48:17.565753 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d1b32-a1f8-4373-b920-564e2a436054 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:48:17.567415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-193d1b32-a1f8-4373-b920-564e2a436054 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:48:17.672392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d1b32-a1f8-4373-b920-564e2a436054 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:48:17.673831 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d1b32-a1f8-4373-b920-564e2a436054 service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:48:17.677579 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:48:17.692121 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 757/1505] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:48:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:48:17.742423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678cbaf-c9e2-40db-96dd-366a023a2676 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:48:17.742928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678cbaf-c9e2-40db-96dd-366a023a2676 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:48:17.743028 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678cbaf-c9e2-40db-96dd-366a023a2676 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:48:17.744253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 749/1506] 10.222.0.13 () {52 vars in 830 bytes} [Mon Aug 8 16:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:48:17.748155 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d1b32-a1f8-4373-b920-564e2a436054 service cinder] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:48:17.749398 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-193d1b32-a1f8-4373-b920-564e2a436054 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:48:17.859385 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d1b32-a1f8-4373-b920-564e2a436054 service cinder] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:48:17.959835 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9fb17-ee17-4b98-84a7-ec0bb144ce45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:48:17.960490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9fb17-ee17-4b98-84a7-ec0bb144ce45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:48:17.960758 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9fb17-ee17-4b98-84a7-ec0bb144ce45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:48:17.961021 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9fb17-ee17-4b98-84a7-ec0bb144ce45 admin admin] PATH_INFO: `/v3/users/7ec6488f8610445989cb432850839a54` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:48:17.974342 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9fb17-ee17-4b98-84a7-ec0bb144ce45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ec6488f8610445989cb432850839a54)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:48:17.975390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: WARNING py.warnings [None req-64e9fb17-ee17-4b98-84a7-ec0bb144ce45 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:48:17.975390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: warnings.warn(msg) Aug 08 16:48:17.975390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: Aug 08 16:48:17.976887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9fb17-ee17-4b98-84a7-ec0bb144ce45 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:48:17.989386 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:48:18.249375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-64e9fb17-ee17-4b98-84a7-ec0bb144ce45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ec6488f8610445989cb432850839a54'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:48:18.351408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-64e9fb17-ee17-4b98-84a7-ec0bb144ce45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ec6488f8610445989cb432850839a54'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:48:18.351836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-64e9fb17-ee17-4b98-84a7-ec0bb144ce45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ec6488f8610445989cb432850839a54'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:48:18.366998 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-64e9fb17-ee17-4b98-84a7-ec0bb144ce45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ec6488f8610445989cb432850839a54'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:48:18.393702 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-64e9fb17-ee17-4b98-84a7-ec0bb144ce45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ec6488f8610445989cb432850839a54'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:48:18.399464 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 758/1507] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:48:17 2022] DELETE /identity/v3/users/7ec6488f8610445989cb432850839a54 => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:48:18.404222 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f88b1-c681-4edb-9f92-52826361512b None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:48:18.404504 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f88b1-c681-4edb-9f92-52826361512b None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:48:18.404777 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f88b1-c681-4edb-9f92-52826361512b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:48:18.466632 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-2b8f88b1-c681-4edb-9f92-52826361512b None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:48:18.467924 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-2b8f88b1-c681-4edb-9f92-52826361512b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:48:18.548371 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:48:18.561149 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 750/1508] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:48:18.571353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9fb17-ee17-4b98-84a7-ec0bb144ce45 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:48:18.572618 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-64e9fb17-ee17-4b98-84a7-ec0bb144ce45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:48:18.675830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9fb17-ee17-4b98-84a7-ec0bb144ce45 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:48:18.732676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:48:18.753115 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037df906-e6de-4396-8e63-8b6d4a3827ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:48:18.753726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037df906-e6de-4396-8e63-8b6d4a3827ad service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:48:18.754225 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037df906-e6de-4396-8e63-8b6d4a3827ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:48:18.754505 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037df906-e6de-4396-8e63-8b6d4a3827ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:48:18.756162 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-037df906-e6de-4396-8e63-8b6d4a3827ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:48:18.867885 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037df906-e6de-4396-8e63-8b6d4a3827ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:48:18.869322 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037df906-e6de-4396-8e63-8b6d4a3827ad service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:48:18.888707 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 759/1509] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:48:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:48:19.388740 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f88b1-c681-4edb-9f92-52826361512b None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:48:19.390032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-2b8f88b1-c681-4edb-9f92-52826361512b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:48:19.401890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f88b1-c681-4edb-9f92-52826361512b None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:48:19.503297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778f778-91af-4db6-a855-e8bf86f511bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:48:19.503932 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778f778-91af-4db6-a855-e8bf86f511bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:48:19.504208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778f778-91af-4db6-a855-e8bf86f511bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:48:19.504478 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778f778-91af-4db6-a855-e8bf86f511bd admin admin] PATH_INFO: `/v3/projects/b9f65f2e0903458aafe194fbd5c83b9c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:48:19.514982 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778f778-91af-4db6-a855-e8bf86f511bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9f65f2e0903458aafe194fbd5c83b9c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:48:19.516025 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: WARNING py.warnings [None req-9778f778-91af-4db6-a855-e8bf86f511bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:48:19.516025 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: warnings.warn(msg) Aug 08 16:48:19.516025 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: Aug 08 16:48:19.517230 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778f778-91af-4db6-a855-e8bf86f511bd admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:48:19.666754 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-9778f778-91af-4db6-a855-e8bf86f511bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9f65f2e0903458aafe194fbd5c83b9c', 'request_id': 'req-9778f778-91af-4db6-a855-e8bf86f511bd'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:48:19.672658 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-9778f778-91af-4db6-a855-e8bf86f511bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9f65f2e0903458aafe194fbd5c83b9c', 'request_id': 'req-9778f778-91af-4db6-a855-e8bf86f511bd'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:48:19.686360 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-9778f778-91af-4db6-a855-e8bf86f511bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9f65f2e0903458aafe194fbd5c83b9c', 'request_id': 'req-9778f778-91af-4db6-a855-e8bf86f511bd'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:48:19.760989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-9778f778-91af-4db6-a855-e8bf86f511bd admin admin] The token cache is being invalidate because project b9f65f2e0903458aafe194fbd5c83b9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:48:19.761274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-9778f778-91af-4db6-a855-e8bf86f511bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:48:19.761697 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-9778f778-91af-4db6-a855-e8bf86f511bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:48:19.763139 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 751/1510] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:48:19 2022] DELETE /identity/v3/projects/b9f65f2e0903458aafe194fbd5c83b9c => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:48:28.886344 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=62 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:48:29.758947 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:49:50.125659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4052e9-5b22-4969-914e-8e127172d96c None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:49:50.126264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4052e9-5b22-4969-914e-8e127172d96c None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:49:50.126264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4052e9-5b22-4969-914e-8e127172d96c None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:49:50.127558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 760/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:49:50.133341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036764f-1561-4973-840b-01368c2bf673 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:49:50.133815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036764f-1561-4973-840b-01368c2bf673 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:49:50.133921 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036764f-1561-4973-840b-01368c2bf673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:49:50.218527 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-6036764f-1561-4973-840b-01368c2bf673 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:49:50.219537 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-6036764f-1561-4973-840b-01368c2bf673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:49:50.325396 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 752/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:49:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:49:50.478954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4052e9-5b22-4969-914e-8e127172d96c None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:49:50.481259 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-fe4052e9-5b22-4969-914e-8e127172d96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:49:50.602617 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4052e9-5b22-4969-914e-8e127172d96c None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:49:50.608132 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bba56e-9238-443f-9b53-039c98349f87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:49:50.608738 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bba56e-9238-443f-9b53-039c98349f87 service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:49:50.609011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bba56e-9238-443f-9b53-039c98349f87 service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:49:50.609281 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bba56e-9238-443f-9b53-039c98349f87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:49:50.617970 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bba56e-9238-443f-9b53-039c98349f87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:49:50.619586 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bba56e-9238-443f-9b53-039c98349f87 service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:49:50.629021 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 761/1513] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:49:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:49:51.464252 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5bdf1-86ab-40ad-a233-461a5ff0ebf4 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:49:51.464841 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5bdf1-86ab-40ad-a233-461a5ff0ebf4 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:49:51.464841 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5bdf1-86ab-40ad-a233-461a5ff0ebf4 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:49:51.466154 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 753/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:49:51.471132 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc19377-838a-4f66-9579-030985c7cd5f None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:49:51.471582 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc19377-838a-4f66-9579-030985c7cd5f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:49:51.471715 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc19377-838a-4f66-9579-030985c7cd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:49:51.494472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-5dc19377-838a-4f66-9579-030985c7cd5f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:49:51.495784 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-5dc19377-838a-4f66-9579-030985c7cd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:49:51.503769 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 762/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:49:51.658456 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5bdf1-86ab-40ad-a233-461a5ff0ebf4 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:49:51.675019 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5bdf1-86ab-40ad-a233-461a5ff0ebf4 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:49:51.680576 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e50a09-7847-4c7c-9e46-2408d46f075e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:49:51.681172 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e50a09-7847-4c7c-9e46-2408d46f075e service nova] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:49:51.681451 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e50a09-7847-4c7c-9e46-2408d46f075e service nova] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:49:51.681728 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e50a09-7847-4c7c-9e46-2408d46f075e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:49:51.690260 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e50a09-7847-4c7c-9e46-2408d46f075e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:49:51.691920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e50a09-7847-4c7c-9e46-2408d46f075e service nova] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:49:51.701878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 754/1516] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:49:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:49:52.641974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ec1e9-95eb-4951-9cc4-63cafbfeb1ff None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:49:52.642537 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ec1e9-95eb-4951-9cc4-63cafbfeb1ff None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:49:52.642537 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ec1e9-95eb-4951-9cc4-63cafbfeb1ff None None] PATH_INFO: `/v3` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:49:52.643912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 763/1517] 10.222.0.23 () {54 vars in 925 bytes} [Mon Aug 8 16:49:52 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:49:52.650922 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6544a9-8740-4500-9ebf-2f8af4c4f864 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:49:52.651341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6544a9-8740-4500-9ebf-2f8af4c4f864 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:49:52.651464 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6544a9-8740-4500-9ebf-2f8af4c4f864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:49:52.712551 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-4a6544a9-8740-4500-9ebf-2f8af4c4f864 None None] MFA Rules not processed for user `6e74dbb121d749749cf5f9d9d7221008`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:49:52.713456 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-4a6544a9-8740-4500-9ebf-2f8af4c4f864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:49:52.731723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:49:52.789580 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 755/1518] 10.222.0.23 () {58 vars in 1015 bytes} [Mon Aug 8 16:49:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:49:56.813632 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0516be-ab5b-46a7-8431-9fc3b2055346 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:49:56.814211 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0516be-ab5b-46a7-8431-9fc3b2055346 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:49:56.814211 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0516be-ab5b-46a7-8431-9fc3b2055346 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:49:56.815530 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 764/1519] 10.222.0.13 () {54 vars in 961 bytes} [Mon Aug 8 16:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:49:56.820860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc424f-0351-4ca7-be26-61a11053e97f None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:49:56.821316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc424f-0351-4ca7-be26-61a11053e97f None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:49:56.821417 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc424f-0351-4ca7-be26-61a11053e97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:49:56.841381 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-45bc424f-0351-4ca7-be26-61a11053e97f None None] MFA Rules not processed for user `6e74dbb121d749749cf5f9d9d7221008`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:49:56.842597 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-45bc424f-0351-4ca7-be26-61a11053e97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:49:56.851750 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 756/1520] 10.222.0.13 () {58 vars in 1058 bytes} [Mon Aug 8 16:49:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:49:56.859590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0516be-ab5b-46a7-8431-9fc3b2055346 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:49:56.873814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0516be-ab5b-46a7-8431-9fc3b2055346 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:49:56.878866 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542fc421-34f9-4672-92c8-c69ea2ff82f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e74dbb121d749749cf5f9d9d7221008', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:49:56.879460 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fc421-34f9-4672-92c8-c69ea2ff82f5 service placement] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:49:56.879736 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fc421-34f9-4672-92c8-c69ea2ff82f5 service placement] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:49:56.880009 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fc421-34f9-4672-92c8-c69ea2ff82f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:49:56.888775 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542fc421-34f9-4672-92c8-c69ea2ff82f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:49:56.890362 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542fc421-34f9-4672-92c8-c69ea2ff82f5 service placement] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:49:56.900940 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 765/1521] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:49:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:50:02.980305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73592d4a-1381-4d3c-8fcc-88e7e7768c53 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:50:02.980887 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73592d4a-1381-4d3c-8fcc-88e7e7768c53 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:50:02.980887 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73592d4a-1381-4d3c-8fcc-88e7e7768c53 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:50:02.982095 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 757/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:50:02.987052 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17746e0-cc20-4ef3-ac79-c3598df50166 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:50:02.987499 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17746e0-cc20-4ef3-ac79-c3598df50166 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:50:02.987643 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17746e0-cc20-4ef3-ac79-c3598df50166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:50:03.008705 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-a17746e0-cc20-4ef3-ac79-c3598df50166 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:50:03.009926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-a17746e0-cc20-4ef3-ac79-c3598df50166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:50:03.018002 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 766/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:50:03.171515 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73592d4a-1381-4d3c-8fcc-88e7e7768c53 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:50:03.187014 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73592d4a-1381-4d3c-8fcc-88e7e7768c53 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:50:03.192424 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd3b34-75e2-49a8-b6ea-c17d801a40bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:50:03.193018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd3b34-75e2-49a8-b6ea-c17d801a40bc service nova] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:50:03.193296 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd3b34-75e2-49a8-b6ea-c17d801a40bc service nova] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:50:03.193562 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd3b34-75e2-49a8-b6ea-c17d801a40bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:50:03.202739 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd3b34-75e2-49a8-b6ea-c17d801a40bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:50:03.206084 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd3b34-75e2-49a8-b6ea-c17d801a40bc service nova] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:50:03.214108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 758/1524] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:50:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:50:04.047482 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21891cd6-ab3c-4267-b077-f2f05038d2b4 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:50:04.048073 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21891cd6-ab3c-4267-b077-f2f05038d2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:50:04.048073 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21891cd6-ab3c-4267-b077-f2f05038d2b4 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:50:04.049409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 767/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:50:04.054411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8150d-1485-4e62-9ebc-60df57fa8779 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:50:04.054866 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8150d-1485-4e62-9ebc-60df57fa8779 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:50:04.054982 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8150d-1485-4e62-9ebc-60df57fa8779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:50:04.074473 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-e8e8150d-1485-4e62-9ebc-60df57fa8779 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:50:04.075720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-e8e8150d-1485-4e62-9ebc-60df57fa8779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:50:04.083814 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 759/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:50:04.238024 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21891cd6-ab3c-4267-b077-f2f05038d2b4 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:50:04.252744 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21891cd6-ab3c-4267-b077-f2f05038d2b4 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:50:04.258976 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6cbd3d-8e0f-4420-a4d0-da6736fcd1e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:50:04.259578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cbd3d-8e0f-4420-a4d0-da6736fcd1e9 service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:50:04.259858 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cbd3d-8e0f-4420-a4d0-da6736fcd1e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:50:04.260126 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cbd3d-8e0f-4420-a4d0-da6736fcd1e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:50:04.268626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6cbd3d-8e0f-4420-a4d0-da6736fcd1e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:50:04.270199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6cbd3d-8e0f-4420-a4d0-da6736fcd1e9 service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:50:04.280856 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 768/1527] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:50:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:50:08.700429 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d1f7c-5552-42d4-9da9-ea4609927791 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:50:08.700980 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d1f7c-5552-42d4-9da9-ea4609927791 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:50:08.700980 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d1f7c-5552-42d4-9da9-ea4609927791 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:50:08.702341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 760/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:50:08.707891 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ab524-4e0d-4b41-ae14-4dadcd23868d None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:50:08.708316 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ab524-4e0d-4b41-ae14-4dadcd23868d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:50:08.708468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ab524-4e0d-4b41-ae14-4dadcd23868d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:50:08.729654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-bf0ab524-4e0d-4b41-ae14-4dadcd23868d None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:50:08.730906 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-bf0ab524-4e0d-4b41-ae14-4dadcd23868d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:50:08.738836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 769/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:50:08.817977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d1f7c-5552-42d4-9da9-ea4609927791 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:50:08.821990 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-b75d1f7c-5552-42d4-9da9-ea4609927791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:50:08.907027 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d1f7c-5552-42d4-9da9-ea4609927791 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:50:08.910703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abe180-1a5c-41da-8000-b0471566c3ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:50:08.911309 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abe180-1a5c-41da-8000-b0471566c3ef service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:50:08.911609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abe180-1a5c-41da-8000-b0471566c3ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:50:08.911879 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abe180-1a5c-41da-8000-b0471566c3ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:50:08.923397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09abe180-1a5c-41da-8000-b0471566c3ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:50:08.925531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09abe180-1a5c-41da-8000-b0471566c3ef service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:50:08.937633 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 761/1530] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:50:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:50:11.009023 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caa69b-908f-4dda-8ab3-7944556f781b None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:50:11.009571 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caa69b-908f-4dda-8ab3-7944556f781b None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:50:11.009571 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caa69b-908f-4dda-8ab3-7944556f781b None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:50:11.010847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 770/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:50:11.016131 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b195a-f70d-40dc-b4aa-817fc7bfaace None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:50:11.016586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b195a-f70d-40dc-b4aa-817fc7bfaace None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:50:11.016701 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b195a-f70d-40dc-b4aa-817fc7bfaace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:50:11.036144 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-744b195a-f70d-40dc-b4aa-817fc7bfaace None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:50:11.037327 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-744b195a-f70d-40dc-b4aa-817fc7bfaace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:50:11.046023 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 762/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:50:11.082832 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5caa69b-908f-4dda-8ab3-7944556f781b None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:50:11.093758 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5caa69b-908f-4dda-8ab3-7944556f781b None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:50:11.097234 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a696a02-9ca2-4101-b025-9bc385c39af8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:50:11.097820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a696a02-9ca2-4101-b025-9bc385c39af8 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:50:11.098090 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a696a02-9ca2-4101-b025-9bc385c39af8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:50:11.098354 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a696a02-9ca2-4101-b025-9bc385c39af8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:50:11.107946 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a696a02-9ca2-4101-b025-9bc385c39af8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:50:11.109346 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a696a02-9ca2-4101-b025-9bc385c39af8 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:50:11.119553 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 771/1533] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:50:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:50:14.565575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7264ff0-726f-4c38-bec2-9db97abc977a None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:50:14.566153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7264ff0-726f-4c38-bec2-9db97abc977a None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:50:14.566153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7264ff0-726f-4c38-bec2-9db97abc977a None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:50:14.567498 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 763/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:50:14.573577 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07877fc4-80a3-47fc-9f27-1bc936a87dee None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:50:14.574031 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07877fc4-80a3-47fc-9f27-1bc936a87dee None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:50:14.574145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07877fc4-80a3-47fc-9f27-1bc936a87dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:50:14.593511 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-07877fc4-80a3-47fc-9f27-1bc936a87dee None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:50:14.594735 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-07877fc4-80a3-47fc-9f27-1bc936a87dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:50:14.603224 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 772/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:50:14.908462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7264ff0-726f-4c38-bec2-9db97abc977a None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:50:14.910121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-f7264ff0-726f-4c38-bec2-9db97abc977a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:50:14.979966 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:50:14.994025 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7264ff0-726f-4c38-bec2-9db97abc977a None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:50:14.998181 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaaf565-abdc-4bf2-b875-61093d033b8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:50:14.998181 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaaf565-abdc-4bf2-b875-61093d033b8d service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:50:14.998449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaaf565-abdc-4bf2-b875-61093d033b8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:50:14.998748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaaf565-abdc-4bf2-b875-61093d033b8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:50:15.007589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaaf565-abdc-4bf2-b875-61093d033b8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:50:15.009035 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaaf565-abdc-4bf2-b875-61093d033b8d service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:50:15.018727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 764/1536] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:50:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:50:17.074397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c062afc-5128-42b9-b7fd-4e1d12da4379 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:50:17.074966 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c062afc-5128-42b9-b7fd-4e1d12da4379 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:50:17.074966 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c062afc-5128-42b9-b7fd-4e1d12da4379 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:50:17.076223 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 773/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:50:17.082151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d3bef-77ae-4e0f-a6c9-7deaf6d7fcbc None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:50:17.082647 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d3bef-77ae-4e0f-a6c9-7deaf6d7fcbc None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:50:17.082970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d3bef-77ae-4e0f-a6c9-7deaf6d7fcbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:50:17.104306 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-f57d3bef-77ae-4e0f-a6c9-7deaf6d7fcbc None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:50:17.105492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-f57d3bef-77ae-4e0f-a6c9-7deaf6d7fcbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:50:17.113290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 765/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:50:17.431880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c062afc-5128-42b9-b7fd-4e1d12da4379 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:50:17.435893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-9c062afc-5128-42b9-b7fd-4e1d12da4379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:50:17.446793 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c062afc-5128-42b9-b7fd-4e1d12da4379 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:50:17.450274 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6342a59-1e81-4612-ae17-f4460f1ceeeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:50:17.450908 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6342a59-1e81-4612-ae17-f4460f1ceeeb service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:50:17.451185 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6342a59-1e81-4612-ae17-f4460f1ceeeb service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:50:17.451447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6342a59-1e81-4612-ae17-f4460f1ceeeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:50:17.461414 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6342a59-1e81-4612-ae17-f4460f1ceeeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:50:17.462864 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6342a59-1e81-4612-ae17-f4460f1ceeeb service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:50:17.472233 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 774/1539] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:50:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:50:27.098377 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=11 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:50:27.468049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=89 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:41.977639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05870bf-17a0-4810-8bf6-930ef9e2d237 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:41.979090 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05870bf-17a0-4810-8bf6-930ef9e2d237 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:41.979090 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05870bf-17a0-4810-8bf6-930ef9e2d237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:42.008335 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-c05870bf-17a0-4810-8bf6-930ef9e2d237 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:42.010160 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-c05870bf-17a0-4810-8bf6-930ef9e2d237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:42.022452 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 766/1540] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:51:42.032040 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9984d7-fb43-497c-90cb-9591a976f543 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:42.033130 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9984d7-fb43-497c-90cb-9591a976f543 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:42.033639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9984d7-fb43-497c-90cb-9591a976f543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:42.070528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-4d9984d7-fb43-497c-90cb-9591a976f543 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:42.072968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-4d9984d7-fb43-497c-90cb-9591a976f543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:42.089642 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 775/1541] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:42.094365 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05870bf-17a0-4810-8bf6-930ef9e2d237 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:42.111187 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05870bf-17a0-4810-8bf6-930ef9e2d237 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:42.120264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac1e01-79d7-4134-8816-57bd40a5d221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:42.120919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac1e01-79d7-4134-8816-57bd40a5d221 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:42.121202 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac1e01-79d7-4134-8816-57bd40a5d221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:42.121473 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac1e01-79d7-4134-8816-57bd40a5d221 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:42.122143 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac1e01-79d7-4134-8816-57bd40a5d221 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:51:42.122446 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac1e01-79d7-4134-8816-57bd40a5d221 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:42.124225 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac1e01-79d7-4134-8816-57bd40a5d221 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:42.149491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 767/1542] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:51:42.158605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9984d7-fb43-497c-90cb-9591a976f543 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:42.172986 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9984d7-fb43-497c-90cb-9591a976f543 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:42.181298 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae8f60-967e-4b62-8294-799286e3d81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:42.182369 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae8f60-967e-4b62-8294-799286e3d81c admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:42.182878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae8f60-967e-4b62-8294-799286e3d81c admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:42.183364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae8f60-967e-4b62-8294-799286e3d81c admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:42.184282 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae8f60-967e-4b62-8294-799286e3d81c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:42.186695 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae8f60-967e-4b62-8294-799286e3d81c admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:42.339340 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 776/1543] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:51:42 2022] POST /identity/v3/projects => generated 375 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:42.343914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac1e01-79d7-4134-8816-57bd40a5d221 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:42.358887 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac1e01-79d7-4134-8816-57bd40a5d221 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:42.362798 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1be5f-a950-486b-baad-a9f87985ea57 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:42.363524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1be5f-a950-486b-baad-a9f87985ea57 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:42.364013 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1be5f-a950-486b-baad-a9f87985ea57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:42.411573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-60b1be5f-a950-486b-baad-a9f87985ea57 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:42.413247 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-60b1be5f-a950-486b-baad-a9f87985ea57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:42.531806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4df6ce-3233-4902-ba7b-b4c26fe83017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:42.532867 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4df6ce-3233-4902-ba7b-b4c26fe83017 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:42.533745 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4df6ce-3233-4902-ba7b-b4c26fe83017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:42.535129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4df6ce-3233-4902-ba7b-b4c26fe83017 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:42.536254 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4df6ce-3233-4902-ba7b-b4c26fe83017 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:42.540483 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4df6ce-3233-4902-ba7b-b4c26fe83017 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:42.574446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 777/1544] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:42.576950 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b69d4-9ca8-473f-bdd0-fcc143a2d780 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:42.577772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b69d4-9ca8-473f-bdd0-fcc143a2d780 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:42.578312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b69d4-9ca8-473f-bdd0-fcc143a2d780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:42.611589 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-138b69d4-9ca8-473f-bdd0-fcc143a2d780 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:42.613164 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-138b69d4-9ca8-473f-bdd0-fcc143a2d780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:42.626705 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 778/1545] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:42.628174 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943f21a-fdbd-484d-9c16-954f341a4db1 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:42.628458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943f21a-fdbd-484d-9c16-954f341a4db1 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:42.628724 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943f21a-fdbd-484d-9c16-954f341a4db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:42.654102 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-d943f21a-fdbd-484d-9c16-954f341a4db1 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:42.655396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-d943f21a-fdbd-484d-9c16-954f341a4db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:42.664360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 779/1546] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:42.666524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c81185-52cc-40bd-af49-6127b8f3340a None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:42.678839 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c81185-52cc-40bd-af49-6127b8f3340a None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:42.678839 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c81185-52cc-40bd-af49-6127b8f3340a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:42.679059 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 768/1547] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:51:42 2022] POST /identity/v3/users => generated 288 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:42.679059 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d872dbd-fe16-4446-a1d0-7178dd897e0d None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:42.679059 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d872dbd-fe16-4446-a1d0-7178dd897e0d None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:42.679059 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d872dbd-fe16-4446-a1d0-7178dd897e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:42.690878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-19c81185-52cc-40bd-af49-6127b8f3340a None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:42.692179 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-19c81185-52cc-40bd-af49-6127b8f3340a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:42.701863 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 780/1548] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:42.713150 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab47f92-c9cc-4025-b2a6-2bb7e4d74f8d None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:42.713150 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab47f92-c9cc-4025-b2a6-2bb7e4d74f8d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:42.713150 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab47f92-c9cc-4025-b2a6-2bb7e4d74f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:42.723857 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-1d872dbd-fe16-4446-a1d0-7178dd897e0d None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:42.728773 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-1d872dbd-fe16-4446-a1d0-7178dd897e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:42.745847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-dab47f92-c9cc-4025-b2a6-2bb7e4d74f8d None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:42.747939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-dab47f92-c9cc-4025-b2a6-2bb7e4d74f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:42.757915 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 769/1549] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:42.760551 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d872dbd-fe16-4446-a1d0-7178dd897e0d None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:42.766710 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 781/1550] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:42.766710 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab47f92-c9cc-4025-b2a6-2bb7e4d74f8d None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:42.788829 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab47f92-c9cc-4025-b2a6-2bb7e4d74f8d None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:42.793798 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a582897-c3cc-4ee2-8afc-557b943bb83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:42.794937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a582897-c3cc-4ee2-8afc-557b943bb83f admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:42.795243 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a582897-c3cc-4ee2-8afc-557b943bb83f admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:42.795702 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a582897-c3cc-4ee2-8afc-557b943bb83f admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:42.796849 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a582897-c3cc-4ee2-8afc-557b943bb83f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:42.800978 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a582897-c3cc-4ee2-8afc-557b943bb83f admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:42.812329 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d872dbd-fe16-4446-a1d0-7178dd897e0d None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:42.818254 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b7713-91b3-44dc-aafd-9ec50a63340d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:42.818974 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b7713-91b3-44dc-aafd-9ec50a63340d admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:42.819277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b7713-91b3-44dc-aafd-9ec50a63340d admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:42.819765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b7713-91b3-44dc-aafd-9ec50a63340d admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:42.820356 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b7713-91b3-44dc-aafd-9ec50a63340d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:51:42.820717 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b7713-91b3-44dc-aafd-9ec50a63340d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:42.823944 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 782/1551] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:42.826493 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17df504-2a1b-4cbf-b703-0714f8b13d44 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:42.827082 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17df504-2a1b-4cbf-b703-0714f8b13d44 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:42.827729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17df504-2a1b-4cbf-b703-0714f8b13d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:42.828602 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b7713-91b3-44dc-aafd-9ec50a63340d admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:42.845181 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 770/1552] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:51:42.846136 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b7713-91b3-44dc-aafd-9ec50a63340d admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:42.857913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b7713-91b3-44dc-aafd-9ec50a63340d admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:42.861274 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-e17df504-2a1b-4cbf-b703-0714f8b13d44 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:42.863383 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-e17df504-2a1b-4cbf-b703-0714f8b13d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:42.863527 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acbb7d-f1d8-44a2-aabb-5349abc7e882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:42.863994 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acbb7d-f1d8-44a2-aabb-5349abc7e882 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:42.864275 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acbb7d-f1d8-44a2-aabb-5349abc7e882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:42.864547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acbb7d-f1d8-44a2-aabb-5349abc7e882 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:42.865199 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acbb7d-f1d8-44a2-aabb-5349abc7e882 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:51:42.865494 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acbb7d-f1d8-44a2-aabb-5349abc7e882 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:42.867164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acbb7d-f1d8-44a2-aabb-5349abc7e882 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:42.881751 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 783/1553] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:42.883151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 771/1554] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:51:42.885325 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b170808-224e-4c30-9f23-827f981ba179 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:42.885618 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b170808-224e-4c30-9f23-827f981ba179 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:42.885878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b170808-224e-4c30-9f23-827f981ba179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:42.890702 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b549d5-9c93-4eae-ba30-badb0dbd3922 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:42.891249 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b549d5-9c93-4eae-ba30-badb0dbd3922 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:42.891731 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b549d5-9c93-4eae-ba30-badb0dbd3922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:42.907565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-9b170808-224e-4c30-9f23-827f981ba179 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:42.908800 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-9b170808-224e-4c30-9f23-827f981ba179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:42.915150 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-77b549d5-9c93-4eae-ba30-badb0dbd3922 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:42.916537 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-77b549d5-9c93-4eae-ba30-badb0dbd3922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:42.917044 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 772/1555] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:42.917887 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b170808-224e-4c30-9f23-827f981ba179 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:42.929142 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 784/1556] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:42.931234 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902a51a-64c6-492f-bcfe-3fe2fefbec09 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:42.931521 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902a51a-64c6-492f-bcfe-3fe2fefbec09 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:42.931803 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902a51a-64c6-492f-bcfe-3fe2fefbec09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:42.937581 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b170808-224e-4c30-9f23-827f981ba179 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:42.942517 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dd316-7a06-466e-ba5c-0d29e1e4c937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:42.943414 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dd316-7a06-466e-ba5c-0d29e1e4c937 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:42.943685 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dd316-7a06-466e-ba5c-0d29e1e4c937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:42.943945 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dd316-7a06-466e-ba5c-0d29e1e4c937 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:42.944580 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83dd316-7a06-466e-ba5c-0d29e1e4c937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:42.946829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83dd316-7a06-466e-ba5c-0d29e1e4c937 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:42.953590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-f902a51a-64c6-492f-bcfe-3fe2fefbec09 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:42.954837 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f902a51a-64c6-492f-bcfe-3fe2fefbec09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:42.957985 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 773/1557] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:42.958902 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dd316-7a06-466e-ba5c-0d29e1e4c937 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:42.965466 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 785/1558] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:42.966867 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b68fd1-fd38-4fba-945e-925e2b20ff0d None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:42.967150 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b68fd1-fd38-4fba-945e-925e2b20ff0d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:42.967424 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b68fd1-fd38-4fba-945e-925e2b20ff0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:42.973099 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dd316-7a06-466e-ba5c-0d29e1e4c937 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:42.978202 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500711b3-3b5a-4e03-a7dc-4aa3b90a487d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:42.978931 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500711b3-3b5a-4e03-a7dc-4aa3b90a487d admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:42.979114 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500711b3-3b5a-4e03-a7dc-4aa3b90a487d admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:42.979380 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500711b3-3b5a-4e03-a7dc-4aa3b90a487d admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:42.980005 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500711b3-3b5a-4e03-a7dc-4aa3b90a487d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:42.981068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: WARNING py.warnings [None req-500711b3-3b5a-4e03-a7dc-4aa3b90a487d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:51:42.981068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: warnings.warn(msg) Aug 08 16:51:42.981068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: Aug 08 16:51:42.982464 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500711b3-3b5a-4e03-a7dc-4aa3b90a487d admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:42.987684 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-e2b68fd1-fd38-4fba-945e-925e2b20ff0d None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:42.988911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-e2b68fd1-fd38-4fba-945e-925e2b20ff0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:42.997808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 786/1559] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:42.998856 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b68fd1-fd38-4fba-945e-925e2b20ff0d None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:43.009661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b68fd1-fd38-4fba-945e-925e2b20ff0d None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:43.014420 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6873f673-8813-495c-984c-5cc010809751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:43.020623 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873f673-8813-495c-984c-5cc010809751 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:43.020623 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873f673-8813-495c-984c-5cc010809751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:43.020623 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873f673-8813-495c-984c-5cc010809751 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:43.020623 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6873f673-8813-495c-984c-5cc010809751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:43.020623 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6873f673-8813-495c-984c-5cc010809751 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:43.075892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 774/1560] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:51:42 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:43.076698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500711b3-3b5a-4e03-a7dc-4aa3b90a487d admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:43.105372 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 787/1561] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:51:42 2022] POST /identity/v3/projects => generated 393 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:43.106704 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6873f673-8813-495c-984c-5cc010809751 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:43.147826 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:43.153041 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500711b3-3b5a-4e03-a7dc-4aa3b90a487d admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:43.155191 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:43.179321 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6873f673-8813-495c-984c-5cc010809751 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:43.184735 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce94c88-a85f-436b-b89f-0c3263d58f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:43.185367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce94c88-a85f-436b-b89f-0c3263d58f67 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:43.185649 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce94c88-a85f-436b-b89f-0c3263d58f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:43.185923 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce94c88-a85f-436b-b89f-0c3263d58f67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:43.186598 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce94c88-a85f-436b-b89f-0c3263d58f67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:51:43.186966 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce94c88-a85f-436b-b89f-0c3263d58f67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:43.187941 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: WARNING py.warnings [None req-0ce94c88-a85f-436b-b89f-0c3263d58f67 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:51:43.187941 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: warnings.warn(msg) Aug 08 16:51:43.187941 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: Aug 08 16:51:43.188843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce94c88-a85f-436b-b89f-0c3263d58f67 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:43.202436 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 788/1562] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:51:43.204205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e8364-07f9-4dc1-8c99-75897e086244 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:43.204472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e8364-07f9-4dc1-8c99-75897e086244 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:43.204746 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e8364-07f9-4dc1-8c99-75897e086244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:43.214617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d936b44-28eb-4109-95a9-62fa38d0bcb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:43.215549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d936b44-28eb-4109-95a9-62fa38d0bcb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:43.215827 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d936b44-28eb-4109-95a9-62fa38d0bcb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:43.216093 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d936b44-28eb-4109-95a9-62fa38d0bcb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:43.216743 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d936b44-28eb-4109-95a9-62fa38d0bcb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:51:43.217045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d936b44-28eb-4109-95a9-62fa38d0bcb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:43.218794 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d936b44-28eb-4109-95a9-62fa38d0bcb6 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:43.226714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-8d5e8364-07f9-4dc1-8c99-75897e086244 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:43.228103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-8d5e8364-07f9-4dc1-8c99-75897e086244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:43.231018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 775/1563] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:51:43.231948 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d936b44-28eb-4109-95a9-62fa38d0bcb6 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:43.243692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 789/1564] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:43.243929 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d936b44-28eb-4109-95a9-62fa38d0bcb6 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:43.245293 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859498c-fad0-4b89-b079-ca1400f04e58 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:43.245988 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859498c-fad0-4b89-b079-ca1400f04e58 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:43.246132 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859498c-fad0-4b89-b079-ca1400f04e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:43.250498 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b6f7d-bf0e-4926-ba72-ded60ea67ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:43.251496 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b6f7d-bf0e-4926-ba72-ded60ea67ab4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:43.251782 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b6f7d-bf0e-4926-ba72-ded60ea67ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:43.252056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b6f7d-bf0e-4926-ba72-ded60ea67ab4 admin admin] PATH_INFO: `/v3/projects/4ab7868e73de46a2afb92c597bb549bc/users/dfa25d3c6a0e4732894b0f6cd7ff4845/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:43.267759 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-c859498c-fad0-4b89-b079-ca1400f04e58 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:43.268518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b6f7d-bf0e-4926-ba72-ded60ea67ab4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ab7868e73de46a2afb92c597bb549bc, user_id=dfa25d3c6a0e4732894b0f6cd7ff4845, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:43.269042 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-c859498c-fad0-4b89-b079-ca1400f04e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:43.269577 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: WARNING py.warnings [None req-2b7b6f7d-bf0e-4926-ba72-ded60ea67ab4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:51:43.269577 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: warnings.warn(msg) Aug 08 16:51:43.269577 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: Aug 08 16:51:43.271876 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b6f7d-bf0e-4926-ba72-ded60ea67ab4 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:43.278063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 790/1565] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:43.279361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df33b7-4292-43b3-98b0-fa9c44a70b61 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:43.279827 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df33b7-4292-43b3-98b0-fa9c44a70b61 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:43.280109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df33b7-4292-43b3-98b0-fa9c44a70b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:43.299387 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-b6df33b7-4292-43b3-98b0-fa9c44a70b61 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:43.300601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b6df33b7-4292-43b3-98b0-fa9c44a70b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:43.309118 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 791/1566] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:43.309926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df33b7-4292-43b3-98b0-fa9c44a70b61 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:43.320043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df33b7-4292-43b3-98b0-fa9c44a70b61 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:43.326307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0e9a7-1f7b-45a3-a4bb-19e23a5f5d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:43.326909 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0e9a7-1f7b-45a3-a4bb-19e23a5f5d8d admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:43.327190 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0e9a7-1f7b-45a3-a4bb-19e23a5f5d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:43.327707 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0e9a7-1f7b-45a3-a4bb-19e23a5f5d8d admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:43.328360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0e9a7-1f7b-45a3-a4bb-19e23a5f5d8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:43.329390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: WARNING py.warnings [None req-dfb0e9a7-1f7b-45a3-a4bb-19e23a5f5d8d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:51:43.329390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: warnings.warn(msg) Aug 08 16:51:43.329390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: Aug 08 16:51:43.330724 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0e9a7-1f7b-45a3-a4bb-19e23a5f5d8d admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:43.362994 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 776/1567] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:51:43 2022] PUT /identity/v3/projects/4ab7868e73de46a2afb92c597bb549bc/users/dfa25d3c6a0e4732894b0f6cd7ff4845/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:51:43.363999 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b6f7d-bf0e-4926-ba72-ded60ea67ab4 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:43.434390 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b6f7d-bf0e-4926-ba72-ded60ea67ab4 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:43.439319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba46fe-f025-4389-9ac1-933278e82924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:43.439954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba46fe-f025-4389-9ac1-933278e82924 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:43.440235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba46fe-f025-4389-9ac1-933278e82924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:43.440508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba46fe-f025-4389-9ac1-933278e82924 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:43.441142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba46fe-f025-4389-9ac1-933278e82924 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:43.442349 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 792/1568] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:51:43 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:43.443288 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0e9a7-1f7b-45a3-a4bb-19e23a5f5d8d admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:43.447606 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba46fe-f025-4389-9ac1-933278e82924 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:43.454911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0e9a7-1f7b-45a3-a4bb-19e23a5f5d8d admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:43.459714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b692acb-89e2-42f7-8b30-33538069ce07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:43.460331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b692acb-89e2-42f7-8b30-33538069ce07 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:43.460604 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b692acb-89e2-42f7-8b30-33538069ce07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:43.460871 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b692acb-89e2-42f7-8b30-33538069ce07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:43.461498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b692acb-89e2-42f7-8b30-33538069ce07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:43.463573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b692acb-89e2-42f7-8b30-33538069ce07 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:43.533431 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 777/1569] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:51:43 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:43.534494 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba46fe-f025-4389-9ac1-933278e82924 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:43.544786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba46fe-f025-4389-9ac1-933278e82924 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:43.553196 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff69bd2-137b-4f4f-af1b-277373c3ea37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:43.553816 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff69bd2-137b-4f4f-af1b-277373c3ea37 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:43.554105 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff69bd2-137b-4f4f-af1b-277373c3ea37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:43.554416 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff69bd2-137b-4f4f-af1b-277373c3ea37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:43.555078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff69bd2-137b-4f4f-af1b-277373c3ea37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:43.557661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff69bd2-137b-4f4f-af1b-277373c3ea37 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:43.563570 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 793/1570] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:51:43 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:43.564394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b692acb-89e2-42f7-8b30-33538069ce07 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:43.629331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b692acb-89e2-42f7-8b30-33538069ce07 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:43.634814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb852112-9dd4-485d-b7ea-a3e772394647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:43.635445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb852112-9dd4-485d-b7ea-a3e772394647 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:43.635723 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb852112-9dd4-485d-b7ea-a3e772394647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:43.635994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb852112-9dd4-485d-b7ea-a3e772394647 admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:43.636647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb852112-9dd4-485d-b7ea-a3e772394647 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:51:43.636945 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb852112-9dd4-485d-b7ea-a3e772394647 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:43.638579 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb852112-9dd4-485d-b7ea-a3e772394647 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:43.646720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 778/1571] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:51:43 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:43.646720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff69bd2-137b-4f4f-af1b-277373c3ea37 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:43.652707 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 794/1572] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:51:43.653727 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb852112-9dd4-485d-b7ea-a3e772394647 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:43.664974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb852112-9dd4-485d-b7ea-a3e772394647 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:43.714364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:43.723925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff69bd2-137b-4f4f-af1b-277373c3ea37 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:43.730138 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea815b32-4f14-45b8-89ec-e56c1a432d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:43.731021 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea815b32-4f14-45b8-89ec-e56c1a432d46 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:43.731464 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea815b32-4f14-45b8-89ec-e56c1a432d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:43.732298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea815b32-4f14-45b8-89ec-e56c1a432d46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:43.733299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea815b32-4f14-45b8-89ec-e56c1a432d46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:51:43.734269 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea815b32-4f14-45b8-89ec-e56c1a432d46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:43.736143 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea815b32-4f14-45b8-89ec-e56c1a432d46 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:43.737291 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6629296-23ef-49f3-827f-5e13e496823b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:43.737963 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6629296-23ef-49f3-827f-5e13e496823b admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:43.738241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6629296-23ef-49f3-827f-5e13e496823b admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:43.738531 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6629296-23ef-49f3-827f-5e13e496823b admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:43.739210 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6629296-23ef-49f3-827f-5e13e496823b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:51:43.739506 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6629296-23ef-49f3-827f-5e13e496823b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:43.741164 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6629296-23ef-49f3-827f-5e13e496823b admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:43.751106 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:43.759651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 779/1573] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:51:43.759651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cd773-609d-4415-9a12-473db0c6f8f7 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:43.759651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cd773-609d-4415-9a12-473db0c6f8f7 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:43.759651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cd773-609d-4415-9a12-473db0c6f8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:43.760340 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 795/1574] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:51:43.760340 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6629296-23ef-49f3-827f-5e13e496823b admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:43.770351 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6629296-23ef-49f3-827f-5e13e496823b admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:43.776653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2516445-ab58-420e-8a05-c32bee114b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:43.777658 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2516445-ab58-420e-8a05-c32bee114b3b admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:43.778338 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2516445-ab58-420e-8a05-c32bee114b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:43.779014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2516445-ab58-420e-8a05-c32bee114b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:43.779700 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2516445-ab58-420e-8a05-c32bee114b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:43.782178 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2516445-ab58-420e-8a05-c32bee114b3b admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:43.794410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-e69cd773-609d-4415-9a12-473db0c6f8f7 None None] MFA Rules not processed for user `dfa25d3c6a0e4732894b0f6cd7ff4845`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:43.795088 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 796/1575] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:43.795681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-e69cd773-609d-4415-9a12-473db0c6f8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:43.796028 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2516445-ab58-420e-8a05-c32bee114b3b admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:43.807734 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2516445-ab58-420e-8a05-c32bee114b3b admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:43.812528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8131cb-88ac-4c7a-b05b-6a01b242dfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:43.813181 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8131cb-88ac-4c7a-b05b-6a01b242dfb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:43.813456 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8131cb-88ac-4c7a-b05b-6a01b242dfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:43.813721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8131cb-88ac-4c7a-b05b-6a01b242dfb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:43.814357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8131cb-88ac-4c7a-b05b-6a01b242dfb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:43.815987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8131cb-88ac-4c7a-b05b-6a01b242dfb8 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:43.826719 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 797/1576] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:43.827858 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8131cb-88ac-4c7a-b05b-6a01b242dfb8 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:43.838699 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8131cb-88ac-4c7a-b05b-6a01b242dfb8 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:43.846660 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912a5ce-8140-4541-b85e-2851a129bc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:43.847283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912a5ce-8140-4541-b85e-2851a129bc3b admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:43.847560 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912a5ce-8140-4541-b85e-2851a129bc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:43.847835 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912a5ce-8140-4541-b85e-2851a129bc3b admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:43.848464 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912a5ce-8140-4541-b85e-2851a129bc3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:43.850393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912a5ce-8140-4541-b85e-2851a129bc3b admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:43.870853 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 780/1577] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:43.871841 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69cd773-609d-4415-9a12-473db0c6f8f7 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:43.886252 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69cd773-609d-4415-9a12-473db0c6f8f7 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:43.888446 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d117c6-f026-4249-9cd7-9c368c8a42a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:43.889143 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d117c6-f026-4249-9cd7-9c368c8a42a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:43.889396 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d117c6-f026-4249-9cd7-9c368c8a42a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:43.889660 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d117c6-f026-4249-9cd7-9c368c8a42a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:43.890294 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d117c6-f026-4249-9cd7-9c368c8a42a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:43.892333 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d117c6-f026-4249-9cd7-9c368c8a42a2 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:43.985201 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 798/1578] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:51:43 2022] POST /identity/v3/users => generated 292 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:43.986012 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912a5ce-8140-4541-b85e-2851a129bc3b admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:43.996822 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912a5ce-8140-4541-b85e-2851a129bc3b admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:44.003374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824eaea3-ef92-438b-bd2b-8b9bc449befa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:44.003992 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824eaea3-ef92-438b-bd2b-8b9bc449befa admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.004270 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824eaea3-ef92-438b-bd2b-8b9bc449befa admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.004544 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824eaea3-ef92-438b-bd2b-8b9bc449befa admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.005210 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824eaea3-ef92-438b-bd2b-8b9bc449befa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:44.007208 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824eaea3-ef92-438b-bd2b-8b9bc449befa admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:44.007511 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 781/1579] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:51:43 2022] POST /identity/v3/users => generated 290 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:44.008377 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d117c6-f026-4249-9cd7-9c368c8a42a2 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:44.018407 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d117c6-f026-4249-9cd7-9c368c8a42a2 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:44.023721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fad419-366c-4130-b68e-66ce5f67001f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:44.024358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fad419-366c-4130-b68e-66ce5f67001f admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.024634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fad419-366c-4130-b68e-66ce5f67001f admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.024899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fad419-366c-4130-b68e-66ce5f67001f admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.025754 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fad419-366c-4130-b68e-66ce5f67001f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:44.027815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fad419-366c-4130-b68e-66ce5f67001f admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:44.087035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 799/1580] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:51:43 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:44.088289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824eaea3-ef92-438b-bd2b-8b9bc449befa admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:44.116327 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 782/1581] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:51:44 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:44.116327 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fad419-366c-4130-b68e-66ce5f67001f admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:44.158880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824eaea3-ef92-438b-bd2b-8b9bc449befa admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:44.185807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fad419-366c-4130-b68e-66ce5f67001f admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:44.190973 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5898a00-14fb-4953-9977-ebc542550d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:44.191595 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5898a00-14fb-4953-9977-ebc542550d05 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.191896 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5898a00-14fb-4953-9977-ebc542550d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.192167 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5898a00-14fb-4953-9977-ebc542550d05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.192815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5898a00-14fb-4953-9977-ebc542550d05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:44.194448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5898a00-14fb-4953-9977-ebc542550d05 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:44.205979 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 783/1582] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:44.206942 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5898a00-14fb-4953-9977-ebc542550d05 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:44.217607 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5898a00-14fb-4953-9977-ebc542550d05 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:44.222275 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5de032-6fed-4edf-a988-31229905fc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:44.222920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5de032-6fed-4edf-a988-31229905fc58 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.223198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5de032-6fed-4edf-a988-31229905fc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.223466 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5de032-6fed-4edf-a988-31229905fc58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.224111 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5de032-6fed-4edf-a988-31229905fc58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:44.225718 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5de032-6fed-4edf-a988-31229905fc58 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:44.226336 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f14b78-b578-482f-9786-a00388e15df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:44.227033 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f14b78-b578-482f-9786-a00388e15df5 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.230055 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f14b78-b578-482f-9786-a00388e15df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.230318 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f14b78-b578-482f-9786-a00388e15df5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.231473 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f14b78-b578-482f-9786-a00388e15df5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:44.234218 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f14b78-b578-482f-9786-a00388e15df5 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:44.237948 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 784/1583] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:44.239412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f1f00-1d87-4d71-853e-2ae0ac3f1989 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.239687 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f1f00-1d87-4d71-853e-2ae0ac3f1989 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.239965 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f1f00-1d87-4d71-853e-2ae0ac3f1989 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.241407 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 785/1584] 10.222.0.13 () {54 vars in 962 bytes} [Mon Aug 8 16:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:51:44.242294 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f1f00-1d87-4d71-853e-2ae0ac3f1989 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:44.255972 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f1f00-1d87-4d71-853e-2ae0ac3f1989 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:44.261484 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a578d-c4c1-4875-bbfe-b735399bb6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:44.262085 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a578d-c4c1-4875-bbfe-b735399bb6ff admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.262357 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a578d-c4c1-4875-bbfe-b735399bb6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.262658 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a578d-c4c1-4875-bbfe-b735399bb6ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.263291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a578d-c4c1-4875-bbfe-b735399bb6ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:44.264898 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a578d-c4c1-4875-bbfe-b735399bb6ff admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:44.271186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:44.278164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 786/1585] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:44.279188 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a578d-c4c1-4875-bbfe-b735399bb6ff admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:44.288820 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a578d-c4c1-4875-bbfe-b735399bb6ff admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:44.294272 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50a3f3-ad7d-468a-89d1-d4a0227bf5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:44.294909 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50a3f3-ad7d-468a-89d1-d4a0227bf5a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.295233 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50a3f3-ad7d-468a-89d1-d4a0227bf5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.295502 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50a3f3-ad7d-468a-89d1-d4a0227bf5a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.296132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd50a3f3-ad7d-468a-89d1-d4a0227bf5a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:44.297716 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd50a3f3-ad7d-468a-89d1-d4a0227bf5a3 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:44.308532 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 787/1586] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:44.309351 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50a3f3-ad7d-468a-89d1-d4a0227bf5a3 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:44.319419 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50a3f3-ad7d-468a-89d1-d4a0227bf5a3 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:44.324417 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f76600-2ceb-4546-94ec-5808b04f4c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:44.325004 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f76600-2ceb-4546-94ec-5808b04f4c3f admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.325277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f76600-2ceb-4546-94ec-5808b04f4c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.325540 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f76600-2ceb-4546-94ec-5808b04f4c3f admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.326151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f76600-2ceb-4546-94ec-5808b04f4c3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:44.328358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f76600-2ceb-4546-94ec-5808b04f4c3f admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:44.368419 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 800/1587] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:51:44 2022] POST /identity/v3/projects => generated 377 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:44.369492 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f14b78-b578-482f-9786-a00388e15df5 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:44.402079 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:44.432248 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f14b78-b578-482f-9786-a00388e15df5 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:44.436807 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fc247-fc15-4d5d-a281-be9193f3fd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:44.437419 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fc247-fc15-4d5d-a281-be9193f3fd16 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.437690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fc247-fc15-4d5d-a281-be9193f3fd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.437995 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fc247-fc15-4d5d-a281-be9193f3fd16 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.438744 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fc247-fc15-4d5d-a281-be9193f3fd16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:44.440649 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fc247-fc15-4d5d-a281-be9193f3fd16 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:44.453166 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 788/1588] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:51:44 2022] POST /identity/v3/users => generated 287 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:44.457707 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f76600-2ceb-4546-94ec-5808b04f4c3f admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:44.464687 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f76600-2ceb-4546-94ec-5808b04f4c3f admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:44.469288 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98350120-3861-4899-bd4a-12f4bff991dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:44.469927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98350120-3861-4899-bd4a-12f4bff991dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.470202 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98350120-3861-4899-bd4a-12f4bff991dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.470469 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98350120-3861-4899-bd4a-12f4bff991dc admin admin] PATH_INFO: `/v3/projects/b2063a114c1942cab80ae4b708be9ab2/users/fdb16e7edbe143e9b96325842896f04e/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.485618 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98350120-3861-4899-bd4a-12f4bff991dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2063a114c1942cab80ae4b708be9ab2, user_id=fdb16e7edbe143e9b96325842896f04e, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:44.488513 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98350120-3861-4899-bd4a-12f4bff991dc admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:44.542705 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 801/1589] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:51:44 2022] POST /identity/v3/users => generated 287 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:44.543522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fc247-fc15-4d5d-a281-be9193f3fd16 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:44.555310 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fc247-fc15-4d5d-a281-be9193f3fd16 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:44.560229 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9985b-8901-4010-8fb2-e60112ff670e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:44.561122 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9985b-8901-4010-8fb2-e60112ff670e admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.561494 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9985b-8901-4010-8fb2-e60112ff670e admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.561770 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9985b-8901-4010-8fb2-e60112ff670e admin admin] PATH_INFO: `/v3/projects/6cac5ca5a8b04bce9a9d8ac938cd35e8/users/75d793a42ec74232899b7a19e36b5867/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.571816 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 789/1590] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:51:44 2022] PUT /identity/v3/projects/b2063a114c1942cab80ae4b708be9ab2/users/fdb16e7edbe143e9b96325842896f04e/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:51:44.575235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f399e8c-4983-4d3c-8199-89f7bba57c36 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.575235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f399e8c-4983-4d3c-8199-89f7bba57c36 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.575235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f399e8c-4983-4d3c-8199-89f7bba57c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.578178 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de9985b-8901-4010-8fb2-e60112ff670e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cac5ca5a8b04bce9a9d8ac938cd35e8, user_id=75d793a42ec74232899b7a19e36b5867, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:44.581012 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de9985b-8901-4010-8fb2-e60112ff670e admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:44.615957 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-1f399e8c-4983-4d3c-8199-89f7bba57c36 None None] MFA Rules not processed for user `9daa7489bced467984d39e8b8ac49d13`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:44.617162 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-1f399e8c-4983-4d3c-8199-89f7bba57c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:44.654180 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 802/1591] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:51:44 2022] PUT /identity/v3/projects/6cac5ca5a8b04bce9a9d8ac938cd35e8/users/75d793a42ec74232899b7a19e36b5867/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:51:44.655085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9985b-8901-4010-8fb2-e60112ff670e admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:44.685287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 790/1592] 10.222.0.13 () {58 vars in 1059 bytes} [Mon Aug 8 16:51:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:44.686050 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f399e8c-4983-4d3c-8199-89f7bba57c36 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:44.722095 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9985b-8901-4010-8fb2-e60112ff670e admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:44.726768 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3f7f1-f11e-428a-a8ad-decf5344ee2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:44.727541 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3f7f1-f11e-428a-a8ad-decf5344ee2c admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.727873 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3f7f1-f11e-428a-a8ad-decf5344ee2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.728143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3f7f1-f11e-428a-a8ad-decf5344ee2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.728808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c3f7f1-f11e-428a-a8ad-decf5344ee2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:44.730396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c3f7f1-f11e-428a-a8ad-decf5344ee2c admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:44.744717 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 803/1593] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:44.745862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3f7f1-f11e-428a-a8ad-decf5344ee2c admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:44.751866 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f399e8c-4983-4d3c-8199-89f7bba57c36 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:44.755675 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3f7f1-f11e-428a-a8ad-decf5344ee2c admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:44.756476 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08603c-e1ae-469d-8899-84191caf728d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:44.757085 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08603c-e1ae-469d-8899-84191caf728d admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.757361 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08603c-e1ae-469d-8899-84191caf728d admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.757637 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08603c-e1ae-469d-8899-84191caf728d admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.758279 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08603c-e1ae-469d-8899-84191caf728d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:44.760650 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08603c-e1ae-469d-8899-84191caf728d admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:44.768749 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecc137-bc47-4606-a35c-ab8e51ee3833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:44.768749 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecc137-bc47-4606-a35c-ab8e51ee3833 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.768749 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecc137-bc47-4606-a35c-ab8e51ee3833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.768749 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecc137-bc47-4606-a35c-ab8e51ee3833 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.769542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecc137-bc47-4606-a35c-ab8e51ee3833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:44.769542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecc137-bc47-4606-a35c-ab8e51ee3833 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:44.771432 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 791/1594] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:44.771432 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08603c-e1ae-469d-8899-84191caf728d admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:44.783876 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08603c-e1ae-469d-8899-84191caf728d admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:44.788601 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d8a7b1-bb5a-4070-9f8e-e073bdf846cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:44.792964 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8a7b1-bb5a-4070-9f8e-e073bdf846cd admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.792964 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8a7b1-bb5a-4070-9f8e-e073bdf846cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.792964 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8a7b1-bb5a-4070-9f8e-e073bdf846cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.792964 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d8a7b1-bb5a-4070-9f8e-e073bdf846cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:44.792964 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d8a7b1-bb5a-4070-9f8e-e073bdf846cd admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:44.802769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 792/1595] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:44.803602 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d8a7b1-bb5a-4070-9f8e-e073bdf846cd admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:44.813782 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d8a7b1-bb5a-4070-9f8e-e073bdf846cd admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:44.818390 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab187ea-d4fe-4a84-b136-d09503ca580e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:44.819022 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab187ea-d4fe-4a84-b136-d09503ca580e admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.819299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab187ea-d4fe-4a84-b136-d09503ca580e admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.819922 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab187ea-d4fe-4a84-b136-d09503ca580e admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.820573 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab187ea-d4fe-4a84-b136-d09503ca580e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:44.822889 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab187ea-d4fe-4a84-b136-d09503ca580e admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:44.828566 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:44.834038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 793/1596] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:44.835358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b151e-5b7d-460e-a8e8-1754a2a5f69b None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.835729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b151e-5b7d-460e-a8e8-1754a2a5f69b None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.836079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b151e-5b7d-460e-a8e8-1754a2a5f69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.855616 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-fa9b151e-5b7d-460e-a8e8-1754a2a5f69b None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:44.856948 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-fa9b151e-5b7d-460e-a8e8-1754a2a5f69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:44.864601 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 794/1597] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:44.865763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1399a51-ac12-4c6b-a705-d47a8cdde9ac None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.866058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1399a51-ac12-4c6b-a705-d47a8cdde9ac None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.866308 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1399a51-ac12-4c6b-a705-d47a8cdde9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.882089 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 804/1598] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:51:44 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:44.882986 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecc137-bc47-4606-a35c-ab8e51ee3833 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:44.889111 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-d1399a51-ac12-4c6b-a705-d47a8cdde9ac None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:44.890316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d1399a51-ac12-4c6b-a705-d47a8cdde9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:44.898195 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 795/1599] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:44.898980 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1399a51-ac12-4c6b-a705-d47a8cdde9ac None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:44.909149 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1399a51-ac12-4c6b-a705-d47a8cdde9ac None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:44.913910 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f9d12-795b-4021-aebc-c6bf8cc96f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:44.914586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f9d12-795b-4021-aebc-c6bf8cc96f6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.914981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f9d12-795b-4021-aebc-c6bf8cc96f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.915263 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f9d12-795b-4021-aebc-c6bf8cc96f6e admin admin] PATH_INFO: `/v3/projects/a8ded1aab7c448aba3df1805e4644c43/users/d447f638293d4d4cb68551e07a397725/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.931268 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f9d12-795b-4021-aebc-c6bf8cc96f6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8ded1aab7c448aba3df1805e4644c43, user_id=d447f638293d4d4cb68551e07a397725, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:44.938800 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f9d12-795b-4021-aebc-c6bf8cc96f6e admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:44.973493 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecc137-bc47-4606-a35c-ab8e51ee3833 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:44.979686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec67d93-eef1-421b-99a5-54c9033fcc20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:44.980319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec67d93-eef1-421b-99a5-54c9033fcc20 service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.980632 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec67d93-eef1-421b-99a5-54c9033fcc20 service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.980899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec67d93-eef1-421b-99a5-54c9033fcc20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.991331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:45.015133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 796/1600] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:51:44 2022] PUT /identity/v3/projects/a8ded1aab7c448aba3df1805e4644c43/users/d447f638293d4d4cb68551e07a397725/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:51:45.016273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f9d12-795b-4021-aebc-c6bf8cc96f6e admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:45.062392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec67d93-eef1-421b-99a5-54c9033fcc20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:45.064046 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec67d93-eef1-421b-99a5-54c9033fcc20 service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:45.074830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 805/1601] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:51:45.075753 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec67d93-eef1-421b-99a5-54c9033fcc20 service nova] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:45.090993 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec67d93-eef1-421b-99a5-54c9033fcc20 service nova] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:45.097096 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a92c94-1dfc-4d76-b761-7157928230e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:45.097721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a92c94-1dfc-4d76-b761-7157928230e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:45.097999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a92c94-1dfc-4d76-b761-7157928230e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:45.098272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a92c94-1dfc-4d76-b761-7157928230e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:45.098965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a92c94-1dfc-4d76-b761-7157928230e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:45.099982 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f9d12-795b-4021-aebc-c6bf8cc96f6e admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:45.100566 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a92c94-1dfc-4d76-b761-7157928230e3 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:45.105484 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622b82b-bbb1-46e6-8625-72076cb27649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:45.106590 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622b82b-bbb1-46e6-8625-72076cb27649 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:45.107147 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622b82b-bbb1-46e6-8625-72076cb27649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:45.107664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622b82b-bbb1-46e6-8625-72076cb27649 admin admin] PATH_INFO: `/v3/projects/e8beb878fb0d489cb0a2cf4a569bf475/users/5891c6b280bd4ad68ef411759fcdec46/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:45.112395 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 806/1602] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:45.116264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a92c94-1dfc-4d76-b761-7157928230e3 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:45.127240 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a92c94-1dfc-4d76-b761-7157928230e3 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:45.128718 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5622b82b-bbb1-46e6-8625-72076cb27649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8beb878fb0d489cb0a2cf4a569bf475, user_id=5891c6b280bd4ad68ef411759fcdec46, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:45.132110 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e09478-7734-415c-b1a1-9f3d95e25869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:45.132500 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5622b82b-bbb1-46e6-8625-72076cb27649 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:45.132726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e09478-7734-415c-b1a1-9f3d95e25869 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:45.133012 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e09478-7734-415c-b1a1-9f3d95e25869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:45.133271 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e09478-7734-415c-b1a1-9f3d95e25869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:45.134539 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e09478-7734-415c-b1a1-9f3d95e25869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:45.136254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e09478-7734-415c-b1a1-9f3d95e25869 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:45.147753 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 807/1603] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:45.150840 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e09478-7734-415c-b1a1-9f3d95e25869 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:45.212043 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 797/1604] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:51:45 2022] PUT /identity/v3/projects/e8beb878fb0d489cb0a2cf4a569bf475/users/5891c6b280bd4ad68ef411759fcdec46/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:51:45.213833 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622b82b-bbb1-46e6-8625-72076cb27649 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:45.228080 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e09478-7734-415c-b1a1-9f3d95e25869 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:45.288783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd1518-9106-409d-9235-d64aee237f7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:45.289554 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd1518-9106-409d-9235-d64aee237f7c service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:45.290561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd1518-9106-409d-9235-d64aee237f7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:45.290934 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd1518-9106-409d-9235-d64aee237f7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:45.301610 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd1518-9106-409d-9235-d64aee237f7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:45.304539 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622b82b-bbb1-46e6-8625-72076cb27649 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:45.304722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd1518-9106-409d-9235-d64aee237f7c service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:45.307408 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c86a6-7f6c-4e05-a880-6effd2ab6eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:45.308037 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c86a6-7f6c-4e05-a880-6effd2ab6eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:45.308320 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c86a6-7f6c-4e05-a880-6effd2ab6eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:45.308595 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c86a6-7f6c-4e05-a880-6effd2ab6eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:45.309246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c86a6-7f6c-4e05-a880-6effd2ab6eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:45.310963 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c86a6-7f6c-4e05-a880-6effd2ab6eb8 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:45.324964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 808/1605] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:51:45.327815 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd1518-9106-409d-9235-d64aee237f7c service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:45.336190 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 798/1606] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:45.338903 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c07d8-2e74-47e1-a25e-c3998ddf7fbf None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:45.340355 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c07d8-2e74-47e1-a25e-c3998ddf7fbf None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:45.340778 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c07d8-2e74-47e1-a25e-c3998ddf7fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:45.341131 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd1518-9106-409d-9235-d64aee237f7c service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:45.348028 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a3074-d1d9-4c8e-8757-f6f1c34b0368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:45.349053 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a3074-d1d9-4c8e-8757-f6f1c34b0368 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:45.349506 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a3074-d1d9-4c8e-8757-f6f1c34b0368 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:45.349979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a3074-d1d9-4c8e-8757-f6f1c34b0368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:45.366345 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a3074-d1d9-4c8e-8757-f6f1c34b0368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:45.368801 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a3074-d1d9-4c8e-8757-f6f1c34b0368 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:45.378777 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-972c07d8-2e74-47e1-a25e-c3998ddf7fbf None None] MFA Rules not processed for user `d447f638293d4d4cb68551e07a397725`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:45.379997 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-972c07d8-2e74-47e1-a25e-c3998ddf7fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:45.381379 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 809/1607] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:51:45.382257 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a3074-d1d9-4c8e-8757-f6f1c34b0368 service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:45.395688 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a3074-d1d9-4c8e-8757-f6f1c34b0368 service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:45.402018 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b466eb-2f98-4c22-81ca-9cc489922b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:45.403060 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b466eb-2f98-4c22-81ca-9cc489922b7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:45.403206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b466eb-2f98-4c22-81ca-9cc489922b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:45.403430 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b466eb-2f98-4c22-81ca-9cc489922b7b admin admin] PATH_INFO: `/v3/projects/00921bd60b1046b4b852832971a86af8/users/65de55053dc84a119bcf9acbe880cdb7/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:45.428085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b466eb-2f98-4c22-81ca-9cc489922b7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=00921bd60b1046b4b852832971a86af8, user_id=65de55053dc84a119bcf9acbe880cdb7, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:45.431811 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b466eb-2f98-4c22-81ca-9cc489922b7b admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:45.450744 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:45.526772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 799/1608] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:45.531119 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c07d8-2e74-47e1-a25e-c3998ddf7fbf None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:45.536302 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 810/1609] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:51:45 2022] PUT /identity/v3/projects/00921bd60b1046b4b852832971a86af8/users/65de55053dc84a119bcf9acbe880cdb7/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:51:45.542438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a633b-111e-4a4a-805b-90b8078ead8c None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:45.543368 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a633b-111e-4a4a-805b-90b8078ead8c None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:45.558804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a633b-111e-4a4a-805b-90b8078ead8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:45.580601 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c07d8-2e74-47e1-a25e-c3998ddf7fbf None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:45.634240 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-e65a633b-111e-4a4a-805b-90b8078ead8c None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:45.636572 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-e65a633b-111e-4a4a-805b-90b8078ead8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:45.646836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:45.699677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8c009-2f51-486b-a0bf-0a495d436937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:45.700508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8c009-2f51-486b-a0bf-0a495d436937 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:45.700826 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8c009-2f51-486b-a0bf-0a495d436937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:45.701111 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8c009-2f51-486b-a0bf-0a495d436937 admin admin] PATH_INFO: `/v3/projects/4cb0de2178b344c3a941e04b29a3a06e/users/18428ac6efe14580b2422c9c2c335efe/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:45.722053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db8c009-2f51-486b-a0bf-0a495d436937 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cb0de2178b344c3a941e04b29a3a06e, user_id=18428ac6efe14580b2422c9c2c335efe, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:45.725894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db8c009-2f51-486b-a0bf-0a495d436937 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:45.735623 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 811/1610] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:45.735623 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a633b-111e-4a4a-805b-90b8078ead8c None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:45.742119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a633b-111e-4a4a-805b-90b8078ead8c None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:45.748370 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fdf86-0c97-40bb-bbf3-5cd6dc39b7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:45.748897 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fdf86-0c97-40bb-bbf3-5cd6dc39b7d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:45.749997 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fdf86-0c97-40bb-bbf3-5cd6dc39b7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:45.749997 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fdf86-0c97-40bb-bbf3-5cd6dc39b7d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:45.750341 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fdf86-0c97-40bb-bbf3-5cd6dc39b7d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:45.751943 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fdf86-0c97-40bb-bbf3-5cd6dc39b7d4 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:45.763949 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 812/1611] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:45.764998 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fdf86-0c97-40bb-bbf3-5cd6dc39b7d4 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:45.827341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 800/1612] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:51:45 2022] PUT /identity/v3/projects/4cb0de2178b344c3a941e04b29a3a06e/users/18428ac6efe14580b2422c9c2c335efe/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:51:45.829716 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf2bad-a4e0-4afe-ab52-d9f8c454fad8 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:45.830345 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf2bad-a4e0-4afe-ab52-d9f8c454fad8 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:45.830935 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf2bad-a4e0-4afe-ab52-d9f8c454fad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:45.883279 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-aebf2bad-a4e0-4afe-ab52-d9f8c454fad8 None None] MFA Rules not processed for user `65de55053dc84a119bcf9acbe880cdb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:45.885108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-aebf2bad-a4e0-4afe-ab52-d9f8c454fad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:45.912419 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fdf86-0c97-40bb-bbf3-5cd6dc39b7d4 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:46.004195 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 801/1613] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:46.005487 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648f682-cc65-4f9b-b234-49175b087131 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:46.007531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648f682-cc65-4f9b-b234-49175b087131 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:46.007822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648f682-cc65-4f9b-b234-49175b087131 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:46.010227 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 802/1614] 10.222.0.13 () {54 vars in 962 bytes} [Mon Aug 8 16:51:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:51:46.011678 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648f682-cc65-4f9b-b234-49175b087131 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:46.013988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d648f682-cc65-4f9b-b234-49175b087131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:46.047859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e10faf-aa7a-42e4-b697-83c88af4b835 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:46.049112 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e10faf-aa7a-42e4-b697-83c88af4b835 service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:46.049586 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e10faf-aa7a-42e4-b697-83c88af4b835 service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:46.050033 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e10faf-aa7a-42e4-b697-83c88af4b835 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:46.118544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:46.130512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e10faf-aa7a-42e4-b697-83c88af4b835 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:46.132237 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e10faf-aa7a-42e4-b697-83c88af4b835 service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:46.139477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648f682-cc65-4f9b-b234-49175b087131 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:46.146068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163fdb11-071b-48bb-baad-5c3407a7fc38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:46.146699 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 813/1615] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:51:46.147733 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e10faf-aa7a-42e4-b697-83c88af4b835 service nova] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:46.148466 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fdb11-071b-48bb-baad-5c3407a7fc38 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:46.149114 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fdb11-071b-48bb-baad-5c3407a7fc38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:46.150210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fdb11-071b-48bb-baad-5c3407a7fc38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:46.220224 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e10faf-aa7a-42e4-b697-83c88af4b835 service nova] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:46.223793 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:46.227389 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ccdaa-6374-492d-9a31-03582259c154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:46.228265 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ccdaa-6374-492d-9a31-03582259c154 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:46.229303 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ccdaa-6374-492d-9a31-03582259c154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:46.229303 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ccdaa-6374-492d-9a31-03582259c154 admin admin] PATH_INFO: `/v3/projects/442f2ca88fce4cbab9c01278239b77ce/users/5a8db3ff64ca44958fd633ba25ee191f/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:46.248721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9ccdaa-6374-492d-9a31-03582259c154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=442f2ca88fce4cbab9c01278239b77ce, user_id=5a8db3ff64ca44958fd633ba25ee191f, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:46.251580 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163fdb11-071b-48bb-baad-5c3407a7fc38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:46.252001 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9ccdaa-6374-492d-9a31-03582259c154 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:46.253135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163fdb11-071b-48bb-baad-5c3407a7fc38 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:46.264923 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 803/1616] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:51:46.266215 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787b4ef-dc99-4eee-87a2-fb88034321ae None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:46.266667 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787b4ef-dc99-4eee-87a2-fb88034321ae None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:46.266938 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787b4ef-dc99-4eee-87a2-fb88034321ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:46.301214 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-4787b4ef-dc99-4eee-87a2-fb88034321ae None None] MFA Rules not processed for user `18428ac6efe14580b2422c9c2c335efe`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:46.302473 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-4787b4ef-dc99-4eee-87a2-fb88034321ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:46.333724 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 814/1617] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:51:46 2022] PUT /identity/v3/projects/442f2ca88fce4cbab9c01278239b77ce/users/5a8db3ff64ca44958fd633ba25ee191f/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:51:46.336124 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90b99f-68df-4b09-be7c-e939465fcfdc None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:46.337276 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90b99f-68df-4b09-be7c-e939465fcfdc None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:46.337900 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90b99f-68df-4b09-be7c-e939465fcfdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:46.376522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-0b90b99f-68df-4b09-be7c-e939465fcfdc None None] MFA Rules not processed for user `9daa7489bced467984d39e8b8ac49d13`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:46.378766 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-0b90b99f-68df-4b09-be7c-e939465fcfdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:46.434313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 804/1618] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:46.438847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcef3a9-37d7-4197-9949-8fd1411e1c0a None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:46.440362 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcef3a9-37d7-4197-9949-8fd1411e1c0a None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:46.445331 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcef3a9-37d7-4197-9949-8fd1411e1c0a None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:46.448364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 805/1619] 10.222.0.13 () {54 vars in 912 bytes} [Mon Aug 8 16:51:46 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:51:46.451164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc305954-d6d2-4756-9d83-bacee14f68dd None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:46.451970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc305954-d6d2-4756-9d83-bacee14f68dd None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:46.452967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc305954-d6d2-4756-9d83-bacee14f68dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:46.496257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-dc305954-d6d2-4756-9d83-bacee14f68dd None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:46.496505 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-dc305954-d6d2-4756-9d83-bacee14f68dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:46.527827 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 815/1620] 10.222.0.13 () {58 vars in 1059 bytes} [Mon Aug 8 16:51:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:46.529235 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90b99f-68df-4b09-be7c-e939465fcfdc None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:46.618003 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:46.624477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 806/1621] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:46.628983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097f52e-bea4-4220-ac95-136ffd552423 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:46.629846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097f52e-bea4-4220-ac95-136ffd552423 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:46.629846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097f52e-bea4-4220-ac95-136ffd552423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:46.639347 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90b99f-68df-4b09-be7c-e939465fcfdc None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:46.648457 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e895f-a561-4e8e-8112-1780a638ae06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:46.650238 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e895f-a561-4e8e-8112-1780a638ae06 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:46.650956 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e895f-a561-4e8e-8112-1780a638ae06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:46.651421 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e895f-a561-4e8e-8112-1780a638ae06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:46.652153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-5097f52e-bea4-4220-ac95-136ffd552423 None None] MFA Rules not processed for user `9daa7489bced467984d39e8b8ac49d13`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:46.652818 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e895f-a561-4e8e-8112-1780a638ae06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:46.653168 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-5097f52e-bea4-4220-ac95-136ffd552423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:46.657096 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e895f-a561-4e8e-8112-1780a638ae06 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:46.663555 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 807/1622] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Aug 8 16:51:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:46.665532 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259177f-f44c-4a57-87c6-95a188a8740a None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:46.665878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259177f-f44c-4a57-87c6-95a188a8740a None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:46.666426 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259177f-f44c-4a57-87c6-95a188a8740a None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:46.668104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 808/1623] 10.222.0.13 () {54 vars in 912 bytes} [Mon Aug 8 16:51:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:51:46.671575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259177f-f44c-4a57-87c6-95a188a8740a None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:46.709785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:46.755773 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259177f-f44c-4a57-87c6-95a188a8740a None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:46.760518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d627859-3d90-4aac-a898-669599896f58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:46.761135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d627859-3d90-4aac-a898-669599896f58 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:46.761414 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d627859-3d90-4aac-a898-669599896f58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:46.761686 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d627859-3d90-4aac-a898-669599896f58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:46.772807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d627859-3d90-4aac-a898-669599896f58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:46.774955 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d627859-3d90-4aac-a898-669599896f58 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:46.786129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 809/1624] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:51:46.797983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf7846-240a-407c-92fc-6a74146853e4 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:46.801511 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf7846-240a-407c-92fc-6a74146853e4 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:46.801511 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf7846-240a-407c-92fc-6a74146853e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:46.816449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 816/1625] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:51:46 2022] POST /identity/v3/projects => generated 371 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:46.818556 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e895f-a561-4e8e-8112-1780a638ae06 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:46.836089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-fcaf7846-240a-407c-92fc-6a74146853e4 None None] MFA Rules not processed for user `9daa7489bced467984d39e8b8ac49d13`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:46.838221 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-fcaf7846-240a-407c-92fc-6a74146853e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:46.956701 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e895f-a561-4e8e-8112-1780a638ae06 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:46.968309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ec604-739a-4435-a127-56756ea496b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:46.970499 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ec604-739a-4435-a127-56756ea496b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:46.974326 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ec604-739a-4435-a127-56756ea496b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:46.974326 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ec604-739a-4435-a127-56756ea496b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:46.981157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 810/1626] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Aug 8 16:51:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:46.982420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf7846-240a-407c-92fc-6a74146853e4 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:47.009803 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ec604-739a-4435-a127-56756ea496b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:47.014710 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ec604-739a-4435-a127-56756ea496b4 service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:47.054635 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 817/1627] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:51:47.057093 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ec604-739a-4435-a127-56756ea496b4 service cinder] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:47.083449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf7846-240a-407c-92fc-6a74146853e4 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:47.083768 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ec604-739a-4435-a127-56756ea496b4 service cinder] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:47.090241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bce32d-f47f-4796-a094-8324a5be5556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:47.090615 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8440f74-fe47-435f-b3fb-a983e1212322 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:47.091448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bce32d-f47f-4796-a094-8324a5be5556 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:47.091898 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bce32d-f47f-4796-a094-8324a5be5556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:47.092090 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8440f74-fe47-435f-b3fb-a983e1212322 service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:47.092409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8440f74-fe47-435f-b3fb-a983e1212322 service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:47.092857 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8440f74-fe47-435f-b3fb-a983e1212322 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:47.093078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bce32d-f47f-4796-a094-8324a5be5556 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:47.094004 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bce32d-f47f-4796-a094-8324a5be5556 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:47.097086 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bce32d-f47f-4796-a094-8324a5be5556 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:47.124445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8440f74-fe47-435f-b3fb-a983e1212322 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:47.128584 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8440f74-fe47-435f-b3fb-a983e1212322 service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:47.162532 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 818/1628] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:51:47.211418 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 811/1629] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:51:46 2022] POST /identity/v3/users => generated 292 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:47.213838 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8440f74-fe47-435f-b3fb-a983e1212322 service cinder] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:47.225560 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8440f74-fe47-435f-b3fb-a983e1212322 service cinder] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:47.231900 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6307c-6169-437f-891a-22a2087c3de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:47.232739 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6307c-6169-437f-891a-22a2087c3de1 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:47.232848 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6307c-6169-437f-891a-22a2087c3de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:47.233073 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6307c-6169-437f-891a-22a2087c3de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:47.233675 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6307c-6169-437f-891a-22a2087c3de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:47.235339 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6307c-6169-437f-891a-22a2087c3de1 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:47.254210 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 819/1630] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:47.256090 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bce32d-f47f-4796-a094-8324a5be5556 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:47.294610 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bce32d-f47f-4796-a094-8324a5be5556 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:47.295267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6307c-6169-437f-891a-22a2087c3de1 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:47.314261 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f300d386-f189-4aa4-94d3-5d7e7f0ff14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:47.314761 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300d386-f189-4aa4-94d3-5d7e7f0ff14b admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:47.315038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300d386-f189-4aa4-94d3-5d7e7f0ff14b admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:47.315302 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300d386-f189-4aa4-94d3-5d7e7f0ff14b admin admin] PATH_INFO: `/v3/projects/1505f06d269d41b3b899f072ba37bfc9/users/b363a8009629476baf4418146e1ebe7f/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:47.329647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6307c-6169-437f-891a-22a2087c3de1 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:47.346893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a40623-822b-49a6-a1e3-16b4b2f6a9d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:47.347589 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a40623-822b-49a6-a1e3-16b4b2f6a9d3 service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:47.347872 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a40623-822b-49a6-a1e3-16b4b2f6a9d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:47.348145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a40623-822b-49a6-a1e3-16b4b2f6a9d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:47.353855 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f300d386-f189-4aa4-94d3-5d7e7f0ff14b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1505f06d269d41b3b899f072ba37bfc9, user_id=b363a8009629476baf4418146e1ebe7f, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:47.356928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f300d386-f189-4aa4-94d3-5d7e7f0ff14b admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:47.464501 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 812/1631] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:51:47 2022] PUT /identity/v3/projects/1505f06d269d41b3b899f072ba37bfc9/users/b363a8009629476baf4418146e1ebe7f/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:51:47.469812 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f300d386-f189-4aa4-94d3-5d7e7f0ff14b admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:47.483150 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a40623-822b-49a6-a1e3-16b4b2f6a9d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:47.484604 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a40623-822b-49a6-a1e3-16b4b2f6a9d3 service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:47.498510 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 820/1632] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:51:47.520351 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:47.587073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f300d386-f189-4aa4-94d3-5d7e7f0ff14b admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:47.593093 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0fa9a-1178-461a-bb0c-67a0cb2f0d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:47.597314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0fa9a-1178-461a-bb0c-67a0cb2f0d6c admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:47.597314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0fa9a-1178-461a-bb0c-67a0cb2f0d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:47.597314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0fa9a-1178-461a-bb0c-67a0cb2f0d6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:47.597314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0fa9a-1178-461a-bb0c-67a0cb2f0d6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:47.597743 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0fa9a-1178-461a-bb0c-67a0cb2f0d6c admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:47.613847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 813/1633] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:47.620146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a40623-822b-49a6-a1e3-16b4b2f6a9d3 service nova] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:47.669245 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a40623-822b-49a6-a1e3-16b4b2f6a9d3 service nova] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:47.689453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf18e8a-42d3-4595-87ec-ed49a96e517f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:47.690148 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf18e8a-42d3-4595-87ec-ed49a96e517f admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:47.690427 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf18e8a-42d3-4595-87ec-ed49a96e517f admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:47.692381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf18e8a-42d3-4595-87ec-ed49a96e517f admin admin] PATH_INFO: `/v3/projects/1505f06d269d41b3b899f072ba37bfc9/users/b363a8009629476baf4418146e1ebe7f/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:47.696880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf18e8a-42d3-4595-87ec-ed49a96e517f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1505f06d269d41b3b899f072ba37bfc9, user_id=b363a8009629476baf4418146e1ebe7f, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:47.700484 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf18e8a-42d3-4595-87ec-ed49a96e517f admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:47.733193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 821/1634] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:51:47 2022] PUT /identity/v3/projects/1505f06d269d41b3b899f072ba37bfc9/users/b363a8009629476baf4418146e1ebe7f/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:51:47.762733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aec4c6-f76d-4599-bcf4-b55a8b483cf9 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:47.763193 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aec4c6-f76d-4599-bcf4-b55a8b483cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:47.763625 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aec4c6-f76d-4599-bcf4-b55a8b483cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:47.814510 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-d7aec4c6-f76d-4599-bcf4-b55a8b483cf9 None None] MFA Rules not processed for user `b363a8009629476baf4418146e1ebe7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:47.816155 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d7aec4c6-f76d-4599-bcf4-b55a8b483cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:47.918009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 814/1635] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:48.623970 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf18e8a-42d3-4595-87ec-ed49a96e517f admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:48.687698 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:48.897614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf18e8a-42d3-4595-87ec-ed49a96e517f admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:48.897614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617a4bf-c0c2-43e4-8002-a0c508bbd28a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:48.897614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617a4bf-c0c2-43e4-8002-a0c508bbd28a service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:48.897614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617a4bf-c0c2-43e4-8002-a0c508bbd28a service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:48.899613 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617a4bf-c0c2-43e4-8002-a0c508bbd28a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:48.899613 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617a4bf-c0c2-43e4-8002-a0c508bbd28a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:48.899613 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617a4bf-c0c2-43e4-8002-a0c508bbd28a service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:48.899613 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 822/1636] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:51:49.265288 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aec4c6-f76d-4599-bcf4-b55a8b483cf9 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:49.352913 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617a4bf-c0c2-43e4-8002-a0c508bbd28a service cinder] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:49.377772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617a4bf-c0c2-43e4-8002-a0c508bbd28a service cinder] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:49.396942 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d5ac5-2443-4711-9217-174db1cfd377 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:49.398818 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d5ac5-2443-4711-9217-174db1cfd377 service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:49.402432 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d5ac5-2443-4711-9217-174db1cfd377 service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:49.403002 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d5ac5-2443-4711-9217-174db1cfd377 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:49.416908 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aec4c6-f76d-4599-bcf4-b55a8b483cf9 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:49.441387 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a7267-1d83-4b46-b98f-e4a9be9741ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:49.441387 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a7267-1d83-4b46-b98f-e4a9be9741ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:49.441387 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a7267-1d83-4b46-b98f-e4a9be9741ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:49.441387 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a7267-1d83-4b46-b98f-e4a9be9741ff admin admin] PATH_INFO: `/v3/users/dfa25d3c6a0e4732894b0f6cd7ff4845` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:49.441721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a7267-1d83-4b46-b98f-e4a9be9741ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfa25d3c6a0e4732894b0f6cd7ff4845)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:49.441721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: WARNING py.warnings [None req-2a7a7267-1d83-4b46-b98f-e4a9be9741ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:51:49.441721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: warnings.warn(msg) Aug 08 16:51:49.441721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: Aug 08 16:51:49.441721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a7267-1d83-4b46-b98f-e4a9be9741ff admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:49.549862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d5ac5-2443-4711-9217-174db1cfd377 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:49.551886 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d5ac5-2443-4711-9217-174db1cfd377 service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:49.581546 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 823/1637] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:51:49.704366 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:49.723134 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-2a7a7267-1d83-4b46-b98f-e4a9be9741ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfa25d3c6a0e4732894b0f6cd7ff4845'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:49.863522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-2a7a7267-1d83-4b46-b98f-e4a9be9741ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfa25d3c6a0e4732894b0f6cd7ff4845'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:49.864899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-2a7a7267-1d83-4b46-b98f-e4a9be9741ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfa25d3c6a0e4732894b0f6cd7ff4845'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:49.897423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-2a7a7267-1d83-4b46-b98f-e4a9be9741ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfa25d3c6a0e4732894b0f6cd7ff4845'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:49.898545 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-2a7a7267-1d83-4b46-b98f-e4a9be9741ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfa25d3c6a0e4732894b0f6cd7ff4845'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:49.988999 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 815/1638] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:51:49 2022] DELETE /identity/v3/users/dfa25d3c6a0e4732894b0f6cd7ff4845 => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:51:49.992813 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3065704-c967-4494-9d26-8878da9b26ef None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:49.994337 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3065704-c967-4494-9d26-8878da9b26ef None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:49.994838 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3065704-c967-4494-9d26-8878da9b26ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:50.028578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-f3065704-c967-4494-9d26-8878da9b26ef None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:50.030353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f3065704-c967-4494-9d26-8878da9b26ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:50.119049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 824/1639] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:50.135538 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a7267-1d83-4b46-b98f-e4a9be9741ff admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:50.137142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-2a7a7267-1d83-4b46-b98f-e4a9be9741ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:50.235132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a7267-1d83-4b46-b98f-e4a9be9741ff admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:50.239704 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff8792-d54b-4bd2-bb36-1a23bbc3eed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:50.240341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff8792-d54b-4bd2-bb36-1a23bbc3eed4 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:50.240974 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff8792-d54b-4bd2-bb36-1a23bbc3eed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:50.240974 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff8792-d54b-4bd2-bb36-1a23bbc3eed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:50.280037 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff8792-d54b-4bd2-bb36-1a23bbc3eed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:50.284322 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff8792-d54b-4bd2-bb36-1a23bbc3eed4 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:50.306832 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 816/1640] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:51:52.343909 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3065704-c967-4494-9d26-8878da9b26ef None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:52.345947 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f3065704-c967-4494-9d26-8878da9b26ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:52.371571 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3065704-c967-4494-9d26-8878da9b26ef None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:52.397355 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb9f4a-8604-42c4-a3d4-0276c9080291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:52.398038 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb9f4a-8604-42c4-a3d4-0276c9080291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:52.398815 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb9f4a-8604-42c4-a3d4-0276c9080291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:52.398815 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb9f4a-8604-42c4-a3d4-0276c9080291 admin admin] PATH_INFO: `/v3/users/65de55053dc84a119bcf9acbe880cdb7` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:52.403012 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb9f4a-8604-42c4-a3d4-0276c9080291 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65de55053dc84a119bcf9acbe880cdb7)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:52.405171 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb9f4a-8604-42c4-a3d4-0276c9080291 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:52.495357 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff8792-d54b-4bd2-bb36-1a23bbc3eed4 service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:52.497524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-59ff8792-d54b-4bd2-bb36-1a23bbc3eed4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:52.527314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff8792-d54b-4bd2-bb36-1a23bbc3eed4 service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:52.553381 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc26c9-be7a-47cf-b6c2-7860fd25efa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:52.554559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc26c9-be7a-47cf-b6c2-7860fd25efa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:52.555841 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc26c9-be7a-47cf-b6c2-7860fd25efa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:52.556705 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc26c9-be7a-47cf-b6c2-7860fd25efa5 admin admin] PATH_INFO: `/v3/projects/4ab7868e73de46a2afb92c597bb549bc` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:52.562343 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc26c9-be7a-47cf-b6c2-7860fd25efa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ab7868e73de46a2afb92c597bb549bc)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:52.568846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc26c9-be7a-47cf-b6c2-7860fd25efa5 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:52.755802 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-45bb9f4a-8604-42c4-a3d4-0276c9080291 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65de55053dc84a119bcf9acbe880cdb7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:52.846685 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-a5cc26c9-be7a-47cf-b6c2-7860fd25efa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ab7868e73de46a2afb92c597bb549bc', 'request_id': 'req-a5cc26c9-be7a-47cf-b6c2-7860fd25efa5'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:52.884587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-a5cc26c9-be7a-47cf-b6c2-7860fd25efa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ab7868e73de46a2afb92c597bb549bc', 'request_id': 'req-a5cc26c9-be7a-47cf-b6c2-7860fd25efa5'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:52.904887 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-a5cc26c9-be7a-47cf-b6c2-7860fd25efa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ab7868e73de46a2afb92c597bb549bc', 'request_id': 'req-a5cc26c9-be7a-47cf-b6c2-7860fd25efa5'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:52.924759 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-45bb9f4a-8604-42c4-a3d4-0276c9080291 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65de55053dc84a119bcf9acbe880cdb7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:52.935108 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-45bb9f4a-8604-42c4-a3d4-0276c9080291 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65de55053dc84a119bcf9acbe880cdb7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:52.945215 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-45bb9f4a-8604-42c4-a3d4-0276c9080291 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65de55053dc84a119bcf9acbe880cdb7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:52.976372 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:52.985207 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-45bb9f4a-8604-42c4-a3d4-0276c9080291 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65de55053dc84a119bcf9acbe880cdb7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:53.004810 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 825/1641] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:51:52 2022] DELETE /identity/v3/users/65de55053dc84a119bcf9acbe880cdb7 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:51:53.013163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdc8bd-ae52-493b-991d-2b313f47c346 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:53.013985 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdc8bd-ae52-493b-991d-2b313f47c346 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:53.016273 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdc8bd-ae52-493b-991d-2b313f47c346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:53.021148 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-a5cc26c9-be7a-47cf-b6c2-7860fd25efa5 admin admin] The token cache is being invalidate because project 4ab7868e73de46a2afb92c597bb549bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:51:53.021652 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-a5cc26c9-be7a-47cf-b6c2-7860fd25efa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:53.026539 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-a5cc26c9-be7a-47cf-b6c2-7860fd25efa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:53.042716 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 817/1642] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:51:52 2022] DELETE /identity/v3/projects/4ab7868e73de46a2afb92c597bb549bc => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:51:53.084544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ed11c-bff4-46bc-b7ed-bb3f88caf88b None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:53.088044 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ed11c-bff4-46bc-b7ed-bb3f88caf88b None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:53.088873 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ed11c-bff4-46bc-b7ed-bb3f88caf88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:53.208867 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-eabdc8bd-ae52-493b-991d-2b313f47c346 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:53.210962 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-6f4ed11c-bff4-46bc-b7ed-bb3f88caf88b None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:53.213939 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-6f4ed11c-bff4-46bc-b7ed-bb3f88caf88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:53.217378 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-eabdc8bd-ae52-493b-991d-2b313f47c346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:53.332861 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:53.376502 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 818/1643] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:53.388966 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b2a73-e12f-4ba7-b232-33631b168155 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:53.389588 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b2a73-e12f-4ba7-b232-33631b168155 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:53.389991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b2a73-e12f-4ba7-b232-33631b168155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:53.427132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-0e9b2a73-e12f-4ba7-b232-33631b168155 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:53.433089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-0e9b2a73-e12f-4ba7-b232-33631b168155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:53.457558 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 819/1644] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:51:53.460761 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b2a73-e12f-4ba7-b232-33631b168155 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:53.469786 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 826/1645] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:53.484166 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b2a73-e12f-4ba7-b232-33631b168155 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:53.496842 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697d944-94f7-4b9b-b14c-9a1339299c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:53.497700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697d944-94f7-4b9b-b14c-9a1339299c82 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:53.498216 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697d944-94f7-4b9b-b14c-9a1339299c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:53.498632 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697d944-94f7-4b9b-b14c-9a1339299c82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:53.499416 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d697d944-94f7-4b9b-b14c-9a1339299c82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:51:53.499821 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d697d944-94f7-4b9b-b14c-9a1339299c82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:53.502594 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d697d944-94f7-4b9b-b14c-9a1339299c82 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:53.510044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabdc8bd-ae52-493b-991d-2b313f47c346 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:53.512252 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-eabdc8bd-ae52-493b-991d-2b313f47c346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:53.521058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 820/1646] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:51:53.526996 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697d944-94f7-4b9b-b14c-9a1339299c82 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:53.551466 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697d944-94f7-4b9b-b14c-9a1339299c82 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:53.564511 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05c543-3e35-4de6-8642-13abea3be860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:53.567053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05c543-3e35-4de6-8642-13abea3be860 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:53.567654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05c543-3e35-4de6-8642-13abea3be860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:53.568833 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05c543-3e35-4de6-8642-13abea3be860 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:53.571441 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c05c543-3e35-4de6-8642-13abea3be860 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:53.575129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c05c543-3e35-4de6-8642-13abea3be860 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:53.646709 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabdc8bd-ae52-493b-991d-2b313f47c346 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:53.654402 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f408e14-544b-467a-be42-5639a0db5d4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:53.655149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f408e14-544b-467a-be42-5639a0db5d4e service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:53.655409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f408e14-544b-467a-be42-5639a0db5d4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:53.656520 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f408e14-544b-467a-be42-5639a0db5d4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:53.681179 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f408e14-544b-467a-be42-5639a0db5d4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:53.683920 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f408e14-544b-467a-be42-5639a0db5d4e service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:53.699410 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 827/1647] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:51:53.703923 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 821/1648] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:51:53 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:53.712236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f408e14-544b-467a-be42-5639a0db5d4e service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:53.726789 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f408e14-544b-467a-be42-5639a0db5d4e service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:53.866677 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0150c-d9d0-44cc-8e77-2fd2d50dee2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:53.868853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0150c-d9d0-44cc-8e77-2fd2d50dee2a admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:53.869245 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0150c-d9d0-44cc-8e77-2fd2d50dee2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:53.870960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0150c-d9d0-44cc-8e77-2fd2d50dee2a admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:53.871667 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c0150c-d9d0-44cc-8e77-2fd2d50dee2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:53.876118 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c0150c-d9d0-44cc-8e77-2fd2d50dee2a admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:54.019987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 828/1649] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:51:53 2022] POST /identity/v3/users => generated 287 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:54.021316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05c543-3e35-4de6-8642-13abea3be860 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:54.036280 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05c543-3e35-4de6-8642-13abea3be860 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:54.041900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d4aec-8d9b-435e-a40e-bfc9b38c5a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:54.045579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d4aec-8d9b-435e-a40e-bfc9b38c5a5d admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:54.045579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d4aec-8d9b-435e-a40e-bfc9b38c5a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:54.045579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d4aec-8d9b-435e-a40e-bfc9b38c5a5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:54.045579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d4aec-8d9b-435e-a40e-bfc9b38c5a5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:54.046453 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d4aec-8d9b-435e-a40e-bfc9b38c5a5d admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:54.058677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 822/1650] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:54.062527 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0150c-d9d0-44cc-8e77-2fd2d50dee2a admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:54.080725 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0150c-d9d0-44cc-8e77-2fd2d50dee2a admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:54.093683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942e6a3-6377-497b-959d-42b398864803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:54.094610 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942e6a3-6377-497b-959d-42b398864803 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:54.094707 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942e6a3-6377-497b-959d-42b398864803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:54.095178 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942e6a3-6377-497b-959d-42b398864803 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:54.095858 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942e6a3-6377-497b-959d-42b398864803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:54.097552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942e6a3-6377-497b-959d-42b398864803 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:54.128961 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d4aec-8d9b-435e-a40e-bfc9b38c5a5d admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:54.129481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 829/1651] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:54.143724 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d4aec-8d9b-435e-a40e-bfc9b38c5a5d admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:54.158279 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26857473-2d3e-480f-a0dd-a87d551ee322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:54.159006 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26857473-2d3e-480f-a0dd-a87d551ee322 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:54.159294 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26857473-2d3e-480f-a0dd-a87d551ee322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:54.159574 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26857473-2d3e-480f-a0dd-a87d551ee322 admin admin] PATH_INFO: `/v3/projects/b49808202b4e4ec89e8d5e3c2f15106b/users/d78234df43c047e18390fa2b803a540e/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:54.193024 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26857473-2d3e-480f-a0dd-a87d551ee322 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b49808202b4e4ec89e8d5e3c2f15106b, user_id=d78234df43c047e18390fa2b803a540e, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:54.195892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26857473-2d3e-480f-a0dd-a87d551ee322 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:54.290222 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 823/1652] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:51:54 2022] PUT /identity/v3/projects/b49808202b4e4ec89e8d5e3c2f15106b/users/d78234df43c047e18390fa2b803a540e/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:51:54.294786 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae6f7e-5f9d-46f4-9a36-f3cadefb308f None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:54.295104 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae6f7e-5f9d-46f4-9a36-f3cadefb308f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:54.295383 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae6f7e-5f9d-46f4-9a36-f3cadefb308f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:54.345297 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-83ae6f7e-5f9d-46f4-9a36-f3cadefb308f None None] MFA Rules not processed for user `d78234df43c047e18390fa2b803a540e`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:54.347240 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-83ae6f7e-5f9d-46f4-9a36-f3cadefb308f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:54.558980 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 830/1653] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:54.559876 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26857473-2d3e-480f-a0dd-a87d551ee322 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:54.751953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26857473-2d3e-480f-a0dd-a87d551ee322 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:54.772093 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004ab6b-2cb4-4225-8101-ceebe8522d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:54.773033 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004ab6b-2cb4-4225-8101-ceebe8522d0c admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:54.773468 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004ab6b-2cb4-4225-8101-ceebe8522d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:54.775220 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004ab6b-2cb4-4225-8101-ceebe8522d0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:54.780489 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6004ab6b-2cb4-4225-8101-ceebe8522d0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:54.780489 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6004ab6b-2cb4-4225-8101-ceebe8522d0c admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:54.919403 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 824/1654] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:51:54 2022] POST /identity/v3/projects => generated 371 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:54.920631 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae6f7e-5f9d-46f4-9a36-f3cadefb308f None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:54.930411 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:54.947266 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae6f7e-5f9d-46f4-9a36-f3cadefb308f None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:55.036169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efc052-533e-46fc-bcd4-c0a6fa19f6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:55.043946 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efc052-533e-46fc-bcd4-c0a6fa19f6ba admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:55.044602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efc052-533e-46fc-bcd4-c0a6fa19f6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:55.044991 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efc052-533e-46fc-bcd4-c0a6fa19f6ba admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:55.046987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99efc052-533e-46fc-bcd4-c0a6fa19f6ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:55.055408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99efc052-533e-46fc-bcd4-c0a6fa19f6ba admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:55.169764 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 831/1655] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:51:54 2022] POST /identity/v3/users => generated 292 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:55.174018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004ab6b-2cb4-4225-8101-ceebe8522d0c admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:55.192412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004ab6b-2cb4-4225-8101-ceebe8522d0c admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:55.217081 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623b5ee-4dfc-4b12-92eb-640183752ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:55.221280 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623b5ee-4dfc-4b12-92eb-640183752ffa admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:55.222707 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623b5ee-4dfc-4b12-92eb-640183752ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:55.223967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623b5ee-4dfc-4b12-92eb-640183752ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:55.241511 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623b5ee-4dfc-4b12-92eb-640183752ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:55.247220 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623b5ee-4dfc-4b12-92eb-640183752ffa admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:55.324590 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 825/1656] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:55.328871 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efc052-533e-46fc-bcd4-c0a6fa19f6ba admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:55.360657 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efc052-533e-46fc-bcd4-c0a6fa19f6ba admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:55.395312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7f6b2-59ec-46d7-a821-315bdbef47db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:55.398149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7f6b2-59ec-46d7-a821-315bdbef47db admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:55.398445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7f6b2-59ec-46d7-a821-315bdbef47db admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:55.398753 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7f6b2-59ec-46d7-a821-315bdbef47db admin admin] PATH_INFO: `/v3/projects/96a56fbd499c4de09af5aa77bf5b9e6e/users/c81b00141fd94e1387c52ff9737d6606/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:55.465970 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7f6b2-59ec-46d7-a821-315bdbef47db admin admin] RBAC: Authorizing `identity:create_grant(project_id=96a56fbd499c4de09af5aa77bf5b9e6e, user_id=c81b00141fd94e1387c52ff9737d6606, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:55.473787 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7f6b2-59ec-46d7-a821-315bdbef47db admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:55.578487 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 832/1657] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:51:55 2022] PUT /identity/v3/projects/96a56fbd499c4de09af5aa77bf5b9e6e/users/c81b00141fd94e1387c52ff9737d6606/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:51:55.579345 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623b5ee-4dfc-4b12-92eb-640183752ffa admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:55.596772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623b5ee-4dfc-4b12-92eb-640183752ffa admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:55.699299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:55.746431 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539fb53-a35f-41c3-a47e-258d5ede64f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:55.748020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539fb53-a35f-41c3-a47e-258d5ede64f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:55.748972 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539fb53-a35f-41c3-a47e-258d5ede64f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:55.750631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539fb53-a35f-41c3-a47e-258d5ede64f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:55.752242 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7539fb53-a35f-41c3-a47e-258d5ede64f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:55.754860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7539fb53-a35f-41c3-a47e-258d5ede64f5 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:55.774828 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7f6b2-59ec-46d7-a821-315bdbef47db admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:55.775141 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 826/1658] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:55.796043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7f6b2-59ec-46d7-a821-315bdbef47db admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:55.800078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539fb53-a35f-41c3-a47e-258d5ede64f5 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:55.809132 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27390d-9734-4fa7-a104-cc15fa93d694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:55.810397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27390d-9734-4fa7-a104-cc15fa93d694 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:55.810593 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-7539fb53-a35f-41c3-a47e-258d5ede64f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:55.811617 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27390d-9734-4fa7-a104-cc15fa93d694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:55.811617 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27390d-9734-4fa7-a104-cc15fa93d694 admin admin] PATH_INFO: `/v3/projects/96a56fbd499c4de09af5aa77bf5b9e6e/users/c81b00141fd94e1387c52ff9737d6606/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:55.813855 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27390d-9734-4fa7-a104-cc15fa93d694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96a56fbd499c4de09af5aa77bf5b9e6e, user_id=c81b00141fd94e1387c52ff9737d6606, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:55.816934 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27390d-9734-4fa7-a104-cc15fa93d694 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:55.857865 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 833/1659] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:51:55 2022] PUT /identity/v3/projects/96a56fbd499c4de09af5aa77bf5b9e6e/users/c81b00141fd94e1387c52ff9737d6606/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:51:55.859879 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c1fd2-bedd-40a5-94d2-cc6fab1eb069 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:55.860229 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c1fd2-bedd-40a5-94d2-cc6fab1eb069 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:55.860541 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c1fd2-bedd-40a5-94d2-cc6fab1eb069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:55.867685 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539fb53-a35f-41c3-a47e-258d5ede64f5 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:55.898077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-5e3c1fd2-bedd-40a5-94d2-cc6fab1eb069 None None] MFA Rules not processed for user `c81b00141fd94e1387c52ff9737d6606`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:55.900355 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-5e3c1fd2-bedd-40a5-94d2-cc6fab1eb069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:56.015802 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bffe64-9dd4-4179-98e8-4510367307c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:56.017013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bffe64-9dd4-4179-98e8-4510367307c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:56.017499 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bffe64-9dd4-4179-98e8-4510367307c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:56.018322 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bffe64-9dd4-4179-98e8-4510367307c6 admin admin] PATH_INFO: `/v3/projects/00921bd60b1046b4b852832971a86af8` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:56.020321 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bffe64-9dd4-4179-98e8-4510367307c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00921bd60b1046b4b852832971a86af8)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:56.024559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bffe64-9dd4-4179-98e8-4510367307c6 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:56.076587 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 834/1660] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:55 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:51:56.089364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c1fd2-bedd-40a5-94d2-cc6fab1eb069 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:56.091286 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-5e3c1fd2-bedd-40a5-94d2-cc6fab1eb069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:56.267872 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c1fd2-bedd-40a5-94d2-cc6fab1eb069 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:56.306484 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-83bffe64-9dd4-4179-98e8-4510367307c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00921bd60b1046b4b852832971a86af8', 'request_id': 'req-83bffe64-9dd4-4179-98e8-4510367307c6'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:56.315036 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-83bffe64-9dd4-4179-98e8-4510367307c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00921bd60b1046b4b852832971a86af8', 'request_id': 'req-83bffe64-9dd4-4179-98e8-4510367307c6'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:56.328082 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-83bffe64-9dd4-4179-98e8-4510367307c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00921bd60b1046b4b852832971a86af8', 'request_id': 'req-83bffe64-9dd4-4179-98e8-4510367307c6'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:56.364833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729c389-9383-4f89-ad3c-073114ef4a61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:56.366481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729c389-9383-4f89-ad3c-073114ef4a61 service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:56.366910 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729c389-9383-4f89-ad3c-073114ef4a61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:56.367307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729c389-9383-4f89-ad3c-073114ef4a61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:56.413813 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-83bffe64-9dd4-4179-98e8-4510367307c6 admin admin] The token cache is being invalidate because project 00921bd60b1046b4b852832971a86af8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:51:56.414075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-83bffe64-9dd4-4179-98e8-4510367307c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:56.415382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-83bffe64-9dd4-4179-98e8-4510367307c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:56.417945 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 827/1661] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:51:55 2022] DELETE /identity/v3/projects/00921bd60b1046b4b852832971a86af8 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:51:56.423039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60cb45-4043-4bd8-ae27-bfa60966ce65 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:56.423484 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60cb45-4043-4bd8-ae27-bfa60966ce65 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:56.423787 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60cb45-4043-4bd8-ae27-bfa60966ce65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:56.481098 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729c389-9383-4f89-ad3c-073114ef4a61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:56.482121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-2e60cb45-4043-4bd8-ae27-bfa60966ce65 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:56.482748 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729c389-9383-4f89-ad3c-073114ef4a61 service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:56.483402 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-2e60cb45-4043-4bd8-ae27-bfa60966ce65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:56.495143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 835/1662] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:56 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:51:56.593736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 828/1663] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:51:56.596120 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:56.602282 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eaee27-e310-4ae4-8820-8ae84d6c1548 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:56.602706 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eaee27-e310-4ae4-8820-8ae84d6c1548 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:56.603217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eaee27-e310-4ae4-8820-8ae84d6c1548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:56.628861 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-19eaee27-e310-4ae4-8820-8ae84d6c1548 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:56.632003 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-19eaee27-e310-4ae4-8820-8ae84d6c1548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:56.649973 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 836/1664] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:56.653772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60cb45-4043-4bd8-ae27-bfa60966ce65 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:56.665887 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60cb45-4043-4bd8-ae27-bfa60966ce65 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:56.689561 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a930c9c-d6c7-4904-91b4-f6e122cfef22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:56.690486 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a930c9c-d6c7-4904-91b4-f6e122cfef22 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:56.690932 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a930c9c-d6c7-4904-91b4-f6e122cfef22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:56.691407 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a930c9c-d6c7-4904-91b4-f6e122cfef22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:56.692232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a930c9c-d6c7-4904-91b4-f6e122cfef22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:51:56.693597 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a930c9c-d6c7-4904-91b4-f6e122cfef22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:56.695460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a930c9c-d6c7-4904-91b4-f6e122cfef22 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:56.753142 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eaee27-e310-4ae4-8820-8ae84d6c1548 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:56.763206 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 829/1665] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:51:56.773440 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eaee27-e310-4ae4-8820-8ae84d6c1548 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:56.779078 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded5342-0339-4930-b55c-5efc304f874f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:56.779373 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded5342-0339-4930-b55c-5efc304f874f admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:56.779652 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded5342-0339-4930-b55c-5efc304f874f admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:56.780058 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded5342-0339-4930-b55c-5efc304f874f admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:56.780709 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded5342-0339-4930-b55c-5efc304f874f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:56.782688 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded5342-0339-4930-b55c-5efc304f874f admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:56.979846 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 837/1666] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:51:56 2022] POST /identity/v3/projects => generated 365 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:56.989915 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a930c9c-d6c7-4904-91b4-f6e122cfef22 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:57.003841 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a930c9c-d6c7-4904-91b4-f6e122cfef22 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:57.161336 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89bd1f-6681-4780-9570-2a00720d7cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:57.164615 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89bd1f-6681-4780-9570-2a00720d7cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:57.164615 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89bd1f-6681-4780-9570-2a00720d7cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:57.164615 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89bd1f-6681-4780-9570-2a00720d7cd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:57.164615 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee89bd1f-6681-4780-9570-2a00720d7cd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:57.167528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee89bd1f-6681-4780-9570-2a00720d7cd5 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:57.259455 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:57.283912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded5342-0339-4930-b55c-5efc304f874f admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:57.284369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 830/1667] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:51:56 2022] POST /identity/v3/users => generated 283 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:57.368668 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded5342-0339-4930-b55c-5efc304f874f admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:57.400509 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3444d1-8163-4813-b720-1b99e3fc9c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:57.401794 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3444d1-8163-4813-b720-1b99e3fc9c68 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:57.402077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3444d1-8163-4813-b720-1b99e3fc9c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:57.402527 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3444d1-8163-4813-b720-1b99e3fc9c68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:57.403364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3444d1-8163-4813-b720-1b99e3fc9c68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:57.405709 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3444d1-8163-4813-b720-1b99e3fc9c68 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:57.457039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 838/1668] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:57.459980 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89bd1f-6681-4780-9570-2a00720d7cd5 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:57.506290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89bd1f-6681-4780-9570-2a00720d7cd5 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:57.540376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b02ce9-32ca-4a7f-b41a-7638ba4871b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:57.545151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b02ce9-32ca-4a7f-b41a-7638ba4871b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:57.545151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b02ce9-32ca-4a7f-b41a-7638ba4871b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:57.545151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b02ce9-32ca-4a7f-b41a-7638ba4871b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:57.546741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b02ce9-32ca-4a7f-b41a-7638ba4871b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:57.547846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b02ce9-32ca-4a7f-b41a-7638ba4871b4 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:57.561022 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 831/1669] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:57.566107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3444d1-8163-4813-b720-1b99e3fc9c68 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:57.614946 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3444d1-8163-4813-b720-1b99e3fc9c68 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:57.655485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fd54e-2382-490b-9e46-3ebb9fc661ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:57.657160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fd54e-2382-490b-9e46-3ebb9fc661ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:57.657960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fd54e-2382-490b-9e46-3ebb9fc661ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:57.658816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fd54e-2382-490b-9e46-3ebb9fc661ff admin admin] PATH_INFO: `/v3/projects/189f7f80f724472a9c6ab308fcb18031/users/7d71f5e92f9e478eb38857b7b8ae4558/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:57.708823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3fd54e-2382-490b-9e46-3ebb9fc661ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=189f7f80f724472a9c6ab308fcb18031, user_id=7d71f5e92f9e478eb38857b7b8ae4558, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:57.715894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3fd54e-2382-490b-9e46-3ebb9fc661ff admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:57.832592 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 839/1670] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:51:57 2022] PUT /identity/v3/projects/189f7f80f724472a9c6ab308fcb18031/users/7d71f5e92f9e478eb38857b7b8ae4558/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:51:57.845863 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac7f46-9423-40aa-8a92-1d6e49527a9f None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:57.846416 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac7f46-9423-40aa-8a92-1d6e49527a9f None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:57.846664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac7f46-9423-40aa-8a92-1d6e49527a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:57.925625 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-f3ac7f46-9423-40aa-8a92-1d6e49527a9f None None] MFA Rules not processed for user `7d71f5e92f9e478eb38857b7b8ae4558`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:57.927215 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-f3ac7f46-9423-40aa-8a92-1d6e49527a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:58.105332 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:58.159222 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 832/1671] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:58.164166 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fd54e-2382-490b-9e46-3ebb9fc661ff admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:58.171505 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac7f46-9423-40aa-8a92-1d6e49527a9f None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:58.173760 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-f3ac7f46-9423-40aa-8a92-1d6e49527a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:58.412945 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac7f46-9423-40aa-8a92-1d6e49527a9f None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:58.429101 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bde85-446a-4de1-a29f-ac6feed71051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:58.430217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bde85-446a-4de1-a29f-ac6feed71051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:58.434288 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bde85-446a-4de1-a29f-ac6feed71051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:58.434547 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fd54e-2382-490b-9e46-3ebb9fc661ff admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:58.435142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bde85-446a-4de1-a29f-ac6feed71051 admin admin] PATH_INFO: `/v3/users/d78234df43c047e18390fa2b803a540e` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:58.436892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07bde85-446a-4de1-a29f-ac6feed71051 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d78234df43c047e18390fa2b803a540e)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:58.440837 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c710f-7925-46e2-927e-3d1fdde32248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:58.441633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c710f-7925-46e2-927e-3d1fdde32248 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:58.442072 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07bde85-446a-4de1-a29f-ac6feed71051 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:58.442246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c710f-7925-46e2-927e-3d1fdde32248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:58.442478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c710f-7925-46e2-927e-3d1fdde32248 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:58.443190 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c710f-7925-46e2-927e-3d1fdde32248 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:58.445116 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c710f-7925-46e2-927e-3d1fdde32248 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:58.590040 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 840/1672] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:51:58 2022] POST /identity/v3/projects => generated 365 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:51:58.596030 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c710f-7925-46e2-927e-3d1fdde32248 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:58.735446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c710f-7925-46e2-927e-3d1fdde32248 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:58.753644 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f07bde85-446a-4de1-a29f-ac6feed71051 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd78234df43c047e18390fa2b803a540e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:58.760299 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ff4f1-2c86-4784-8abc-ffb181f15b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:58.761579 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ff4f1-2c86-4784-8abc-ffb181f15b05 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:58.761853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ff4f1-2c86-4784-8abc-ffb181f15b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:58.762140 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ff4f1-2c86-4784-8abc-ffb181f15b05 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:58.763499 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167ff4f1-2c86-4784-8abc-ffb181f15b05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:58.776799 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167ff4f1-2c86-4784-8abc-ffb181f15b05 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:58.939225 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f07bde85-446a-4de1-a29f-ac6feed71051 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd78234df43c047e18390fa2b803a540e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:58.939721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f07bde85-446a-4de1-a29f-ac6feed71051 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd78234df43c047e18390fa2b803a540e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:58.965465 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f07bde85-446a-4de1-a29f-ac6feed71051 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd78234df43c047e18390fa2b803a540e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:58.965465 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f07bde85-446a-4de1-a29f-ac6feed71051 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd78234df43c047e18390fa2b803a540e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:51:59.017029 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 841/1673] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:51:58 2022] POST /identity/v3/users => generated 289 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:51:59.017029 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74853ff4-b49d-46b3-86dc-2979ba1b3ba0 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:59.017288 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74853ff4-b49d-46b3-86dc-2979ba1b3ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:59.017541 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74853ff4-b49d-46b3-86dc-2979ba1b3ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:59.052012 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 833/1674] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:51:58 2022] DELETE /identity/v3/users/d78234df43c047e18390fa2b803a540e => generated 0 bytes in 878 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:51:59.052012 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bde85-446a-4de1-a29f-ac6feed71051 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:59.057204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-f07bde85-446a-4de1-a29f-ac6feed71051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:59.108429 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-74853ff4-b49d-46b3-86dc-2979ba1b3ba0 None None] MFA Rules not processed for user `fdb16e7edbe143e9b96325842896f04e`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:59.109984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-74853ff4-b49d-46b3-86dc-2979ba1b3ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:59.298920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bde85-446a-4de1-a29f-ac6feed71051 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:59.345526 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9197e0-524c-4baf-875a-5f3cb30a04b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:59.346174 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9197e0-524c-4baf-875a-5f3cb30a04b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:59.346703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9197e0-524c-4baf-875a-5f3cb30a04b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:59.346990 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9197e0-524c-4baf-875a-5f3cb30a04b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:59.347651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9197e0-524c-4baf-875a-5f3cb30a04b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:59.351516 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9197e0-524c-4baf-875a-5f3cb30a04b1 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:59.370971 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 842/1675] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:51:59.373991 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b81c76-a1d3-4b4a-9178-f46284dc789f None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:59.374861 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b81c76-a1d3-4b4a-9178-f46284dc789f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:59.375899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b81c76-a1d3-4b4a-9178-f46284dc789f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:59.387238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 834/1676] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:51:59.390520 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9197e0-524c-4baf-875a-5f3cb30a04b1 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:59.396459 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-7d9197e0-524c-4baf-875a-5f3cb30a04b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:59.442178 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-92b81c76-a1d3-4b4a-9178-f46284dc789f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:59.455003 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-92b81c76-a1d3-4b4a-9178-f46284dc789f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:59.504002 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 843/1677] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:51:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:59.505029 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b81c76-a1d3-4b4a-9178-f46284dc789f None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:59.538232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b81c76-a1d3-4b4a-9178-f46284dc789f None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:59.553837 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b695653-97cb-4f01-b626-921d74a1d1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:59.556428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b695653-97cb-4f01-b626-921d74a1d1ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:59.556428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b695653-97cb-4f01-b626-921d74a1d1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:59.556428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b695653-97cb-4f01-b626-921d74a1d1ae admin admin] PATH_INFO: `/v3/projects/088eb806b7c64badafcc97acce9d56b8/users/cb7ae194a9cf46e1802270ba9d90e114/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:59.608272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b695653-97cb-4f01-b626-921d74a1d1ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=088eb806b7c64badafcc97acce9d56b8, user_id=cb7ae194a9cf46e1802270ba9d90e114, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:59.609725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:59.620367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b695653-97cb-4f01-b626-921d74a1d1ae admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:59.657648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:59.694316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9197e0-524c-4baf-875a-5f3cb30a04b1 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:59.732409 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2a257-9d9b-44eb-ada9-3d5373df9469 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:59.732409 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2a257-9d9b-44eb-ada9-3d5373df9469 service nova] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:59.732409 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2a257-9d9b-44eb-ada9-3d5373df9469 service nova] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:59.732409 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2a257-9d9b-44eb-ada9-3d5373df9469 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:59.743936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 844/1678] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:51:59 2022] PUT /identity/v3/projects/088eb806b7c64badafcc97acce9d56b8/users/cb7ae194a9cf46e1802270ba9d90e114/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:51:59.744663 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b695653-97cb-4f01-b626-921d74a1d1ae admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:59.748981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-0b695653-97cb-4f01-b626-921d74a1d1ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:59.752475 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2a257-9d9b-44eb-ada9-3d5373df9469 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:59.760231 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2a257-9d9b-44eb-ada9-3d5373df9469 service nova] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:59.791679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 835/1679] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:51:59.791679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2a257-9d9b-44eb-ada9-3d5373df9469 service nova] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:59.821869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2a257-9d9b-44eb-ada9-3d5373df9469 service nova] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:59.935706 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b695653-97cb-4f01-b626-921d74a1d1ae admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:59.952840 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbad71-70fc-4bdd-a590-c046f6bba350 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:59.953492 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbad71-70fc-4bdd-a590-c046f6bba350 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:59.953772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbad71-70fc-4bdd-a590-c046f6bba350 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:59.954044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbad71-70fc-4bdd-a590-c046f6bba350 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:00.018670 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237d64b-7f3e-44b0-bc77-2bf21d41a114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:00.019696 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237d64b-7f3e-44b0-bc77-2bf21d41a114 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:00.019986 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237d64b-7f3e-44b0-bc77-2bf21d41a114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:00.020254 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237d64b-7f3e-44b0-bc77-2bf21d41a114 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:00.020901 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b237d64b-7f3e-44b0-bc77-2bf21d41a114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:00.022521 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b237d64b-7f3e-44b0-bc77-2bf21d41a114 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:00.046546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 836/1680] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:00.055410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237d64b-7f3e-44b0-bc77-2bf21d41a114 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:00.072125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237d64b-7f3e-44b0-bc77-2bf21d41a114 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:00.089002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2d860-d5d1-4c37-baa4-e1a64bac19ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:00.090257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2d860-d5d1-4c37-baa4-e1a64bac19ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:00.090727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2d860-d5d1-4c37-baa4-e1a64bac19ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:00.091160 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2d860-d5d1-4c37-baa4-e1a64bac19ec admin admin] PATH_INFO: `/v3/projects/088eb806b7c64badafcc97acce9d56b8/users/cb7ae194a9cf46e1802270ba9d90e114/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:00.094261 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2d860-d5d1-4c37-baa4-e1a64bac19ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=088eb806b7c64badafcc97acce9d56b8, user_id=cb7ae194a9cf46e1802270ba9d90e114, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:00.100668 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2d860-d5d1-4c37-baa4-e1a64bac19ec admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:00.137356 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 837/1681] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:52:00 2022] PUT /identity/v3/projects/088eb806b7c64badafcc97acce9d56b8/users/cb7ae194a9cf46e1802270ba9d90e114/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:00.143553 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbad71-70fc-4bdd-a590-c046f6bba350 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:00.146311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbad71-70fc-4bdd-a590-c046f6bba350 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:00.153612 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e09b3b-e3a8-408d-a6e2-3679c3888922 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:00.154054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e09b3b-e3a8-408d-a6e2-3679c3888922 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:00.154468 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e09b3b-e3a8-408d-a6e2-3679c3888922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:00.181678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 845/1682] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:00.197897 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d28248-f1f4-41b5-8441-fb6047cd3e52 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:00.198186 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d28248-f1f4-41b5-8441-fb6047cd3e52 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:00.198462 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d28248-f1f4-41b5-8441-fb6047cd3e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:00.210373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-63e09b3b-e3a8-408d-a6e2-3679c3888922 None None] MFA Rules not processed for user `cb7ae194a9cf46e1802270ba9d90e114`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:00.214708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-63e09b3b-e3a8-408d-a6e2-3679c3888922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:00.275924 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-46d28248-f1f4-41b5-8441-fb6047cd3e52 None None] MFA Rules not processed for user `75d793a42ec74232899b7a19e36b5867`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:00.279212 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-46d28248-f1f4-41b5-8441-fb6047cd3e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:00.336714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 838/1683] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:00 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:52:00.361546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e09b3b-e3a8-408d-a6e2-3679c3888922 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:00.367105 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-63e09b3b-e3a8-408d-a6e2-3679c3888922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:00.485531 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 846/1684] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:52:00.492803 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4ad93-75b1-4a30-8235-dac48ac52173 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:00.493672 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4ad93-75b1-4a30-8235-dac48ac52173 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:00.493672 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4ad93-75b1-4a30-8235-dac48ac52173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:00.522365 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e09b3b-e3a8-408d-a6e2-3679c3888922 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:00.526145 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc2e92-e471-49ec-b333-5982dae52685 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:00.527000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc2e92-e471-49ec-b333-5982dae52685 service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:00.527331 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc2e92-e471-49ec-b333-5982dae52685 service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:00.527604 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc2e92-e471-49ec-b333-5982dae52685 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:00.540182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc2e92-e471-49ec-b333-5982dae52685 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:00.541932 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc2e92-e471-49ec-b333-5982dae52685 service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:00.568280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-04b4ad93-75b1-4a30-8235-dac48ac52173 None None] MFA Rules not processed for user `75d793a42ec74232899b7a19e36b5867`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:00.568751 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:00.569566 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-04b4ad93-75b1-4a30-8235-dac48ac52173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:00.597855 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 847/1685] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:00.604167 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 839/1686] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:00 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:00.606814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4ad93-75b1-4a30-8235-dac48ac52173 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:00.831955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4ad93-75b1-4a30-8235-dac48ac52173 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:00.855909 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac6c11-c515-4ece-babd-00b4aae8f94d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:00.856581 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac6c11-c515-4ece-babd-00b4aae8f94d service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:00.856883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac6c11-c515-4ece-babd-00b4aae8f94d service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:00.857149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac6c11-c515-4ece-babd-00b4aae8f94d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:00.901993 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac6c11-c515-4ece-babd-00b4aae8f94d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:00.904066 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac6c11-c515-4ece-babd-00b4aae8f94d service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:00.915793 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:00.944979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 848/1687] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:00 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:01.347065 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc2e92-e471-49ec-b333-5982dae52685 service cinder] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:01.347839 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-efbc2e92-e471-49ec-b333-5982dae52685 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:01.504425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc2e92-e471-49ec-b333-5982dae52685 service cinder] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:01.518347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcfb17-df8a-4515-b4a1-a16a5b7dc2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:01.519095 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcfb17-df8a-4515-b4a1-a16a5b7dc2c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:01.519538 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcfb17-df8a-4515-b4a1-a16a5b7dc2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:01.524556 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcfb17-df8a-4515-b4a1-a16a5b7dc2c8 admin admin] PATH_INFO: `/v3/users/d447f638293d4d4cb68551e07a397725` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:01.524556 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dcfb17-df8a-4515-b4a1-a16a5b7dc2c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d447f638293d4d4cb68551e07a397725)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:01.529147 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dcfb17-df8a-4515-b4a1-a16a5b7dc2c8 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:01.767970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-34dcfb17-df8a-4515-b4a1-a16a5b7dc2c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd447f638293d4d4cb68551e07a397725'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:01.872373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-34dcfb17-df8a-4515-b4a1-a16a5b7dc2c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd447f638293d4d4cb68551e07a397725'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:01.877936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-34dcfb17-df8a-4515-b4a1-a16a5b7dc2c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd447f638293d4d4cb68551e07a397725'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:01.891201 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac6c11-c515-4ece-babd-00b4aae8f94d service nova] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:01.893224 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ffac6c11-c515-4ece-babd-00b4aae8f94d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:01.913814 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-34dcfb17-df8a-4515-b4a1-a16a5b7dc2c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd447f638293d4d4cb68551e07a397725'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:01.917333 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-34dcfb17-df8a-4515-b4a1-a16a5b7dc2c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd447f638293d4d4cb68551e07a397725'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:01.964579 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac6c11-c515-4ece-babd-00b4aae8f94d service nova] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:01.979253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd71780-5bf5-4b58-883e-493699476628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:01.979934 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd71780-5bf5-4b58-883e-493699476628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:01.980219 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd71780-5bf5-4b58-883e-493699476628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:01.980566 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd71780-5bf5-4b58-883e-493699476628 admin admin] PATH_INFO: `/v3/projects/b49808202b4e4ec89e8d5e3c2f15106b` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:01.987731 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd71780-5bf5-4b58-883e-493699476628 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b49808202b4e4ec89e8d5e3c2f15106b)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:01.989129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: WARNING py.warnings [None req-acd71780-5bf5-4b58-883e-493699476628 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:52:01.989129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: warnings.warn(msg) Aug 08 16:52:01.989129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: Aug 08 16:52:01.996916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd71780-5bf5-4b58-883e-493699476628 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:02.000690 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 840/1688] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:52:01 2022] DELETE /identity/v3/users/d447f638293d4d4cb68551e07a397725 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:02.002682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9788e-2507-4d76-a01c-903435b55fad None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:02.006492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9788e-2507-4d76-a01c-903435b55fad None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:02.007348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9788e-2507-4d76-a01c-903435b55fad None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:02.017822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 841/1689] 10.222.0.13 () {54 vars in 912 bytes} [Mon Aug 8 16:52:02 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:52:02.019484 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d958a0-1e39-4a73-bc38-7fffd50e2784 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:02.023269 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d958a0-1e39-4a73-bc38-7fffd50e2784 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:02.025334 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d958a0-1e39-4a73-bc38-7fffd50e2784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:02.090039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-c5d958a0-1e39-4a73-bc38-7fffd50e2784 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:02.091762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-c5d958a0-1e39-4a73-bc38-7fffd50e2784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:02.277047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 842/1690] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:02.279268 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bc8b3-b2d6-4d32-8794-e20ad6fb0bc0 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:02.284107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-acd71780-5bf5-4b58-883e-493699476628 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b49808202b4e4ec89e8d5e3c2f15106b', 'request_id': 'req-acd71780-5bf5-4b58-883e-493699476628'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:02.301515 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bc8b3-b2d6-4d32-8794-e20ad6fb0bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:02.301515 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bc8b3-b2d6-4d32-8794-e20ad6fb0bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:02.373800 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-acd71780-5bf5-4b58-883e-493699476628 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b49808202b4e4ec89e8d5e3c2f15106b', 'request_id': 'req-acd71780-5bf5-4b58-883e-493699476628'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:02.453712 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-acd71780-5bf5-4b58-883e-493699476628 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b49808202b4e4ec89e8d5e3c2f15106b', 'request_id': 'req-acd71780-5bf5-4b58-883e-493699476628'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:02.507001 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-acd71780-5bf5-4b58-883e-493699476628 admin admin] The token cache is being invalidate because project b49808202b4e4ec89e8d5e3c2f15106b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:52:02.511423 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-acd71780-5bf5-4b58-883e-493699476628 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:02.512830 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-447bc8b3-b2d6-4d32-8794-e20ad6fb0bc0 None None] MFA Rules not processed for user `9482bf89c1b648c1a91ee00ee5fbba63`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:02.513786 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-acd71780-5bf5-4b58-883e-493699476628 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:02.517196 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-447bc8b3-b2d6-4d32-8794-e20ad6fb0bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:02.520690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 849/1691] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:52:01 2022] DELETE /identity/v3/projects/b49808202b4e4ec89e8d5e3c2f15106b => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:52:02.520690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd71780-5bf5-4b58-883e-493699476628 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:02.522917 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-acd71780-5bf5-4b58-883e-493699476628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:02.668855 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:02.717366 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 843/1692] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Aug 8 16:52:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:02.720299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447bc8b3-b2d6-4d32-8794-e20ad6fb0bc0 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:02.724008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-447bc8b3-b2d6-4d32-8794-e20ad6fb0bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:02.764363 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd71780-5bf5-4b58-883e-493699476628 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:02.777233 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41481287-56a8-436f-9c57-471bc51684b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:02.782896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41481287-56a8-436f-9c57-471bc51684b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:02.783276 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41481287-56a8-436f-9c57-471bc51684b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:02.784767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41481287-56a8-436f-9c57-471bc51684b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:02.795008 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-41481287-56a8-436f-9c57-471bc51684b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:02.840882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:02.960084 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447bc8b3-b2d6-4d32-8794-e20ad6fb0bc0 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:02.981438 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12e1f7-dbf0-45c6-b34b-111a89bddb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:02.982379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12e1f7-dbf0-45c6-b34b-111a89bddb51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:02.983358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12e1f7-dbf0-45c6-b34b-111a89bddb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:02.983807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12e1f7-dbf0-45c6-b34b-111a89bddb51 admin admin] PATH_INFO: `/v3/users/c81b00141fd94e1387c52ff9737d6606` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:02.985545 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce12e1f7-dbf0-45c6-b34b-111a89bddb51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c81b00141fd94e1387c52ff9737d6606)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:02.988180 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce12e1f7-dbf0-45c6-b34b-111a89bddb51 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:03.014474 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41481287-56a8-436f-9c57-471bc51684b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:03.018290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41481287-56a8-436f-9c57-471bc51684b8 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:03.053470 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 850/1693] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:03.056352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41481287-56a8-436f-9c57-471bc51684b8 service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:03.061115 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-41481287-56a8-436f-9c57-471bc51684b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:03.091527 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41481287-56a8-436f-9c57-471bc51684b8 service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:03.114383 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf0a9f-8fe2-4023-a4cf-1c58fb52ec9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:03.116760 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf0a9f-8fe2-4023-a4cf-1c58fb52ec9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:03.117225 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf0a9f-8fe2-4023-a4cf-1c58fb52ec9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:03.127369 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf0a9f-8fe2-4023-a4cf-1c58fb52ec9e admin admin] PATH_INFO: `/v3/users/7d71f5e92f9e478eb38857b7b8ae4558` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:03.136192 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf0a9f-8fe2-4023-a4cf-1c58fb52ec9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d71f5e92f9e478eb38857b7b8ae4558)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:03.142051 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf0a9f-8fe2-4023-a4cf-1c58fb52ec9e admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:03.239026 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-ce12e1f7-dbf0-45c6-b34b-111a89bddb51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c81b00141fd94e1387c52ff9737d6606'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:03.312478 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-ce12e1f7-dbf0-45c6-b34b-111a89bddb51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c81b00141fd94e1387c52ff9737d6606'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:03.317120 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-ce12e1f7-dbf0-45c6-b34b-111a89bddb51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c81b00141fd94e1387c52ff9737d6606'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:03.323205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-ce12e1f7-dbf0-45c6-b34b-111a89bddb51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c81b00141fd94e1387c52ff9737d6606'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:03.326383 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-ce12e1f7-dbf0-45c6-b34b-111a89bddb51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c81b00141fd94e1387c52ff9737d6606'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:03.376452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-d8bf0a9f-8fe2-4023-a4cf-1c58fb52ec9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d71f5e92f9e478eb38857b7b8ae4558'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:03.413899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 844/1694] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:52:02 2022] DELETE /identity/v3/users/c81b00141fd94e1387c52ff9737d6606 => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:03.426802 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12e1f7-dbf0-45c6-b34b-111a89bddb51 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:03.438482 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ce12e1f7-dbf0-45c6-b34b-111a89bddb51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:03.487229 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-d8bf0a9f-8fe2-4023-a4cf-1c58fb52ec9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d71f5e92f9e478eb38857b7b8ae4558'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:03.488550 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-d8bf0a9f-8fe2-4023-a4cf-1c58fb52ec9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d71f5e92f9e478eb38857b7b8ae4558'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:03.501340 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-d8bf0a9f-8fe2-4023-a4cf-1c58fb52ec9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d71f5e92f9e478eb38857b7b8ae4558'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:03.552304 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-d8bf0a9f-8fe2-4023-a4cf-1c58fb52ec9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d71f5e92f9e478eb38857b7b8ae4558'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:03.563782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 851/1695] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:52:03 2022] DELETE /identity/v3/users/7d71f5e92f9e478eb38857b7b8ae4558 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:03.567677 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e67d9b-a67d-4226-bdb8-8b97d106eda2 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:03.567976 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e67d9b-a67d-4226-bdb8-8b97d106eda2 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:03.568256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e67d9b-a67d-4226-bdb8-8b97d106eda2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:03.609437 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12e1f7-dbf0-45c6-b34b-111a89bddb51 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:03.667363 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-95e67d9b-a67d-4226-bdb8-8b97d106eda2 None None] MFA Rules not processed for user `5a8db3ff64ca44958fd633ba25ee191f`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:03.668635 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-95e67d9b-a67d-4226-bdb8-8b97d106eda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:03.736664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed84b6-d22a-4158-a5d9-185f13079ef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:03.740073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed84b6-d22a-4158-a5d9-185f13079ef2 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:03.740073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed84b6-d22a-4158-a5d9-185f13079ef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:03.740073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed84b6-d22a-4158-a5d9-185f13079ef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:03.741780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-bfed84b6-d22a-4158-a5d9-185f13079ef2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:03.755689 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed84b6-d22a-4158-a5d9-185f13079ef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:03.759184 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed84b6-d22a-4158-a5d9-185f13079ef2 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:03.771808 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 845/1696] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:03.773671 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1d675-9968-4a55-9439-96ec6be52b71 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:03.774882 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1d675-9968-4a55-9439-96ec6be52b71 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:03.775731 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1d675-9968-4a55-9439-96ec6be52b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:03.804068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-6da1d675-9968-4a55-9439-96ec6be52b71 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:03.806759 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-6da1d675-9968-4a55-9439-96ec6be52b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:03.838234 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 852/1697] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:03.851625 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e67d9b-a67d-4226-bdb8-8b97d106eda2 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:03.854088 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-95e67d9b-a67d-4226-bdb8-8b97d106eda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:03.968296 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 846/1698] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:03.982141 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1d675-9968-4a55-9439-96ec6be52b71 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:03.983834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-6da1d675-9968-4a55-9439-96ec6be52b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:03.987209 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:04.007753 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1d675-9968-4a55-9439-96ec6be52b71 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:04.020872 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e5ba62-28a3-4a24-842c-fa57d21cefb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:04.022763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5ba62-28a3-4a24-842c-fa57d21cefb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:04.022763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5ba62-28a3-4a24-842c-fa57d21cefb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:04.022763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5ba62-28a3-4a24-842c-fa57d21cefb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:04.030015 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e67d9b-a67d-4226-bdb8-8b97d106eda2 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:04.043607 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a1c15-a423-4071-a9a7-fefe1c21971c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:04.045987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a1c15-a423-4071-a9a7-fefe1c21971c service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:04.047287 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a1c15-a423-4071-a9a7-fefe1c21971c service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:04.047287 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a1c15-a423-4071-a9a7-fefe1c21971c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:04.047724 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e5ba62-28a3-4a24-842c-fa57d21cefb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:04.049214 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e5ba62-28a3-4a24-842c-fa57d21cefb4 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:04.067576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94a1c15-a423-4071-a9a7-fefe1c21971c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:04.069211 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94a1c15-a423-4071-a9a7-fefe1c21971c service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:04.082212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 847/1699] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:52:04.096930 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 853/1700] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:52:04.339083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e5ba62-28a3-4a24-842c-fa57d21cefb4 service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:04.340589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-04e5ba62-28a3-4a24-842c-fa57d21cefb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:04.363227 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e5ba62-28a3-4a24-842c-fa57d21cefb4 service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:04.375497 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd94fc-e496-4a65-abf2-997797a5c949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:04.376543 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd94fc-e496-4a65-abf2-997797a5c949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:04.377126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd94fc-e496-4a65-abf2-997797a5c949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:04.377661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd94fc-e496-4a65-abf2-997797a5c949 admin admin] PATH_INFO: `/v3/projects/a8ded1aab7c448aba3df1805e4644c43` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:04.379277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd94fc-e496-4a65-abf2-997797a5c949 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8ded1aab7c448aba3df1805e4644c43)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:04.381546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd94fc-e496-4a65-abf2-997797a5c949 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:04.607014 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c5fd94fc-e496-4a65-abf2-997797a5c949 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8ded1aab7c448aba3df1805e4644c43', 'request_id': 'req-c5fd94fc-e496-4a65-abf2-997797a5c949'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:04.654962 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c5fd94fc-e496-4a65-abf2-997797a5c949 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8ded1aab7c448aba3df1805e4644c43', 'request_id': 'req-c5fd94fc-e496-4a65-abf2-997797a5c949'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:04.687882 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c5fd94fc-e496-4a65-abf2-997797a5c949 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8ded1aab7c448aba3df1805e4644c43', 'request_id': 'req-c5fd94fc-e496-4a65-abf2-997797a5c949'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:04.725643 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a1c15-a423-4071-a9a7-fefe1c21971c service nova] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:04.728202 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-e94a1c15-a423-4071-a9a7-fefe1c21971c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:04.753456 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a1c15-a423-4071-a9a7-fefe1c21971c service nova] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:04.804174 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c5fd94fc-e496-4a65-abf2-997797a5c949 admin admin] The token cache is being invalidate because project a8ded1aab7c448aba3df1805e4644c43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:52:04.804485 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c5fd94fc-e496-4a65-abf2-997797a5c949 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:04.815081 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c5fd94fc-e496-4a65-abf2-997797a5c949 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:04.817617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 848/1701] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:52:04 2022] DELETE /identity/v3/projects/a8ded1aab7c448aba3df1805e4644c43 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:04.824233 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36142d6d-43b0-4b63-8f24-a65aed7968a3 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:04.828223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36142d6d-43b0-4b63-8f24-a65aed7968a3 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:04.828223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36142d6d-43b0-4b63-8f24-a65aed7968a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:04.908938 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a27ff5-b175-4b5c-b529-870a8b7af995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:04.909934 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a27ff5-b175-4b5c-b529-870a8b7af995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:04.914659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a27ff5-b175-4b5c-b529-870a8b7af995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:04.914659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a27ff5-b175-4b5c-b529-870a8b7af995 admin admin] PATH_INFO: `/v3/projects/96a56fbd499c4de09af5aa77bf5b9e6e` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:04.914659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a27ff5-b175-4b5c-b529-870a8b7af995 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96a56fbd499c4de09af5aa77bf5b9e6e)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:04.917149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a27ff5-b175-4b5c-b529-870a8b7af995 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:04.946614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-36142d6d-43b0-4b63-8f24-a65aed7968a3 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:04.951714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-36142d6d-43b0-4b63-8f24-a65aed7968a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:04.966524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:04.994767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 849/1702] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:05.007260 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371246bb-993f-4282-add4-b68b356be50c None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:05.007740 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371246bb-993f-4282-add4-b68b356be50c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:05.014506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371246bb-993f-4282-add4-b68b356be50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:05.101568 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-371246bb-993f-4282-add4-b68b356be50c None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:05.108965 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-371246bb-993f-4282-add4-b68b356be50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:05.113952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-c7a27ff5-b175-4b5c-b529-870a8b7af995 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96a56fbd499c4de09af5aa77bf5b9e6e', 'request_id': 'req-c7a27ff5-b175-4b5c-b529-870a8b7af995'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:05.230197 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-c7a27ff5-b175-4b5c-b529-870a8b7af995 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96a56fbd499c4de09af5aa77bf5b9e6e', 'request_id': 'req-c7a27ff5-b175-4b5c-b529-870a8b7af995'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:05.243433 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-c7a27ff5-b175-4b5c-b529-870a8b7af995 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96a56fbd499c4de09af5aa77bf5b9e6e', 'request_id': 'req-c7a27ff5-b175-4b5c-b529-870a8b7af995'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:05.281008 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-c7a27ff5-b175-4b5c-b529-870a8b7af995 admin admin] The token cache is being invalidate because project 96a56fbd499c4de09af5aa77bf5b9e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:52:05.281303 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-c7a27ff5-b175-4b5c-b529-870a8b7af995 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:05.286018 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-c7a27ff5-b175-4b5c-b529-870a8b7af995 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:05.287775 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 854/1703] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:52:04 2022] DELETE /identity/v3/projects/96a56fbd499c4de09af5aa77bf5b9e6e => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:52:05.289339 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e01e1-457b-44f6-b007-cb49f62eac1c None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:05.290189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e01e1-457b-44f6-b007-cb49f62eac1c None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:05.290783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e01e1-457b-44f6-b007-cb49f62eac1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:05.297394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 850/1704] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:52:05.300926 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac15f4-f3f6-42b8-a94d-f2ffb0679814 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:05.300926 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac15f4-f3f6-42b8-a94d-f2ffb0679814 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:05.300926 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac15f4-f3f6-42b8-a94d-f2ffb0679814 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:05.302860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 851/1705] 10.222.0.13 () {54 vars in 961 bytes} [Mon Aug 8 16:52:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:52:05.304365 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd80d99-ba2b-4905-877d-eef1bfe0c71f None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:05.307410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd80d99-ba2b-4905-877d-eef1bfe0c71f None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:05.307772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd80d99-ba2b-4905-877d-eef1bfe0c71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:05.337553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-9cd80d99-ba2b-4905-877d-eef1bfe0c71f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:05.338898 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-9cd80d99-ba2b-4905-877d-eef1bfe0c71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:05.350265 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-b75e01e1-457b-44f6-b007-cb49f62eac1c None None] MFA Rules not processed for user `5891c6b280bd4ad68ef411759fcdec46`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:05.351588 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b75e01e1-457b-44f6-b007-cb49f62eac1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:05.360219 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 852/1706] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:05.360219 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd80d99-ba2b-4905-877d-eef1bfe0c71f None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:05.374864 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd80d99-ba2b-4905-877d-eef1bfe0c71f None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:05.382831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6f9dc-6836-48ae-bc77-e0e333b4578f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:05.387258 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6f9dc-6836-48ae-bc77-e0e333b4578f admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:05.387258 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6f9dc-6836-48ae-bc77-e0e333b4578f admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:05.387258 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6f9dc-6836-48ae-bc77-e0e333b4578f admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:05.387258 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6f9dc-6836-48ae-bc77-e0e333b4578f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:52:05.387258 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6f9dc-6836-48ae-bc77-e0e333b4578f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:05.388188 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6f9dc-6836-48ae-bc77-e0e333b4578f admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:05.411255 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 853/1707] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:52:05.419219 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1242ee-308e-45b0-b837-fa3b5cc1ba26 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:05.419219 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1242ee-308e-45b0-b837-fa3b5cc1ba26 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:05.419219 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1242ee-308e-45b0-b837-fa3b5cc1ba26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:05.449631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-fb1242ee-308e-45b0-b837-fa3b5cc1ba26 None None] MFA Rules not processed for user `6e74dbb121d749749cf5f9d9d7221008`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:05.456238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-fb1242ee-308e-45b0-b837-fa3b5cc1ba26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:05.493267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 855/1708] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:05.499859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64152a-382a-48f3-89f4-5f8b9950b370 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:05.500188 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64152a-382a-48f3-89f4-5f8b9950b370 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:05.500490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64152a-382a-48f3-89f4-5f8b9950b370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:05.530250 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-4a64152a-382a-48f3-89f4-5f8b9950b370 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:05.532639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-4a64152a-382a-48f3-89f4-5f8b9950b370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:05.548797 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 856/1709] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:05.548797 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64152a-382a-48f3-89f4-5f8b9950b370 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:05.584057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64152a-382a-48f3-89f4-5f8b9950b370 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:05.596551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3806648-28e4-435d-820f-974854365157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:05.597521 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3806648-28e4-435d-820f-974854365157 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:05.599386 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3806648-28e4-435d-820f-974854365157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:05.600094 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3806648-28e4-435d-820f-974854365157 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:05.601210 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3806648-28e4-435d-820f-974854365157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:05.602200 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 854/1710] 10.222.0.13 () {58 vars in 1058 bytes} [Mon Aug 8 16:52:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:05.603765 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3806648-28e4-435d-820f-974854365157 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:05.604095 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade68e55-8909-4e19-bcb5-4a4cf8ff043c None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:05.604378 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade68e55-8909-4e19-bcb5-4a4cf8ff043c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:05.604702 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade68e55-8909-4e19-bcb5-4a4cf8ff043c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:05.634850 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-ade68e55-8909-4e19-bcb5-4a4cf8ff043c None None] MFA Rules not processed for user `5891c6b280bd4ad68ef411759fcdec46`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:05.636104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ade68e55-8909-4e19-bcb5-4a4cf8ff043c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:05.646534 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 855/1711] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:05.647481 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade68e55-8909-4e19-bcb5-4a4cf8ff043c None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:05.673717 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade68e55-8909-4e19-bcb5-4a4cf8ff043c None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:05.681434 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa1576-53c1-4e51-aaa5-befdbe7aaad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:05.682205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa1576-53c1-4e51-aaa5-befdbe7aaad1 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:05.682523 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa1576-53c1-4e51-aaa5-befdbe7aaad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:05.683332 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa1576-53c1-4e51-aaa5-befdbe7aaad1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:05.684083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa1576-53c1-4e51-aaa5-befdbe7aaad1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:52:05.684391 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa1576-53c1-4e51-aaa5-befdbe7aaad1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:05.686758 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa1576-53c1-4e51-aaa5-befdbe7aaad1 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:05.715199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 857/1712] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:52:05 2022] POST /identity/v3/projects => generated 365 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:52:05.717568 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba98a46-a460-4cd1-bbf7-bf2e2d7540f0 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:05.718183 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba98a46-a460-4cd1-bbf7-bf2e2d7540f0 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:05.719686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba98a46-a460-4cd1-bbf7-bf2e2d7540f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:05.721914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 856/1713] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:52:05.724105 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa1576-53c1-4e51-aaa5-befdbe7aaad1 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:05.749649 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa1576-53c1-4e51-aaa5-befdbe7aaad1 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:05.759143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-fba98a46-a460-4cd1-bbf7-bf2e2d7540f0 None None] MFA Rules not processed for user `75d793a42ec74232899b7a19e36b5867`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:05.762582 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-fba98a46-a460-4cd1-bbf7-bf2e2d7540f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:05.820724 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:05.897452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 858/1714] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:05.900232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba98a46-a460-4cd1-bbf7-bf2e2d7540f0 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:05.902976 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-fba98a46-a460-4cd1-bbf7-bf2e2d7540f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:05.923760 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667cec96-696d-49f2-a98b-958c07a24674 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e74dbb121d749749cf5f9d9d7221008', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:05.924392 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cec96-696d-49f2-a98b-958c07a24674 service placement] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:05.924665 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cec96-696d-49f2-a98b-958c07a24674 service placement] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:05.924937 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cec96-696d-49f2-a98b-958c07a24674 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:05.934257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-667cec96-696d-49f2-a98b-958c07a24674 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:05.960666 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667cec96-696d-49f2-a98b-958c07a24674 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:05.963936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667cec96-696d-49f2-a98b-958c07a24674 service placement] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:05.992260 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 857/1715] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:05.993498 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667cec96-696d-49f2-a98b-958c07a24674 service placement] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:06.121165 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba98a46-a460-4cd1-bbf7-bf2e2d7540f0 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:06.128235 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd65f47-7839-48dc-9de2-eb3da0efccba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:06.129629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd65f47-7839-48dc-9de2-eb3da0efccba service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:06.130009 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd65f47-7839-48dc-9de2-eb3da0efccba service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:06.130706 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd65f47-7839-48dc-9de2-eb3da0efccba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:06.181718 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667cec96-696d-49f2-a98b-958c07a24674 service placement] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:06.203241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b94bb-5de6-47a2-9f0f-343d5e28e484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:06.203403 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b94bb-5de6-47a2-9f0f-343d5e28e484 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:06.203658 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b94bb-5de6-47a2-9f0f-343d5e28e484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:06.203931 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b94bb-5de6-47a2-9f0f-343d5e28e484 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:06.205200 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b94bb-5de6-47a2-9f0f-343d5e28e484 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:06.209724 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b94bb-5de6-47a2-9f0f-343d5e28e484 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:06.282971 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd65f47-7839-48dc-9de2-eb3da0efccba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:06.286215 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd65f47-7839-48dc-9de2-eb3da0efccba service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:06.309144 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 859/1716] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:06.310253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd65f47-7839-48dc-9de2-eb3da0efccba service nova] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:06.352869 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd65f47-7839-48dc-9de2-eb3da0efccba service nova] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:06.357658 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c106e03-7e8c-4f6a-8895-5205551250dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:06.358310 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c106e03-7e8c-4f6a-8895-5205551250dd admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:06.358646 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c106e03-7e8c-4f6a-8895-5205551250dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:06.358925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c106e03-7e8c-4f6a-8895-5205551250dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:06.360394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c106e03-7e8c-4f6a-8895-5205551250dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:06.366524 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c106e03-7e8c-4f6a-8895-5205551250dd admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:06.370451 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 858/1717] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:52:05 2022] POST /identity/v3/users => generated 283 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:52:06.375113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b94bb-5de6-47a2-9f0f-343d5e28e484 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:06.424382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b94bb-5de6-47a2-9f0f-343d5e28e484 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:06.453165 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e73d4-b572-4865-b58b-1c6c37c79251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:06.453165 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e73d4-b572-4865-b58b-1c6c37c79251 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:06.453165 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e73d4-b572-4865-b58b-1c6c37c79251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:06.453562 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e73d4-b572-4865-b58b-1c6c37c79251 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:06.454151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e73d4-b572-4865-b58b-1c6c37c79251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:06.458430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e73d4-b572-4865-b58b-1c6c37c79251 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:06.462697 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 860/1718] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:52:06 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:52:06.462697 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c106e03-7e8c-4f6a-8895-5205551250dd admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:06.477492 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-5c106e03-7e8c-4f6a-8895-5205551250dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:06.493169 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 859/1719] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:06.494209 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e73d4-b572-4865-b58b-1c6c37c79251 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:06.532324 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e73d4-b572-4865-b58b-1c6c37c79251 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:06.683117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:06.736430 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c106e03-7e8c-4f6a-8895-5205551250dd admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:06.745548 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96399fc5-5b96-42fc-a208-677ec4a366e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:06.746226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96399fc5-5b96-42fc-a208-677ec4a366e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:06.746703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96399fc5-5b96-42fc-a208-677ec4a366e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:06.746987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96399fc5-5b96-42fc-a208-677ec4a366e5 admin admin] PATH_INFO: `/v3/projects/189f7f80f724472a9c6ab308fcb18031` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:06.748352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96399fc5-5b96-42fc-a208-677ec4a366e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=189f7f80f724472a9c6ab308fcb18031)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:06.750311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96399fc5-5b96-42fc-a208-677ec4a366e5 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:06.814322 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6d618-c33d-4568-b179-819fa899cbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:06.815077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6d618-c33d-4568-b179-819fa899cbe8 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:06.815421 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6d618-c33d-4568-b179-819fa899cbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:06.815756 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6d618-c33d-4568-b179-819fa899cbe8 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:06.816720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d6d618-c33d-4568-b179-819fa899cbe8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:06.819803 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d6d618-c33d-4568-b179-819fa899cbe8 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:06.918163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-96399fc5-5b96-42fc-a208-677ec4a366e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '189f7f80f724472a9c6ab308fcb18031', 'request_id': 'req-96399fc5-5b96-42fc-a208-677ec4a366e5'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:06.957845 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 860/1720] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:52:06 2022] POST /identity/v3/users => generated 293 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:52:06.959210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6d618-c33d-4568-b179-819fa899cbe8 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:06.989069 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6d618-c33d-4568-b179-819fa899cbe8 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:07.052950 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-96399fc5-5b96-42fc-a208-677ec4a366e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '189f7f80f724472a9c6ab308fcb18031', 'request_id': 'req-96399fc5-5b96-42fc-a208-677ec4a366e5'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:07.064997 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-96399fc5-5b96-42fc-a208-677ec4a366e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '189f7f80f724472a9c6ab308fcb18031', 'request_id': 'req-96399fc5-5b96-42fc-a208-677ec4a366e5'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:07.106473 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-96399fc5-5b96-42fc-a208-677ec4a366e5 admin admin] The token cache is being invalidate because project 189f7f80f724472a9c6ab308fcb18031 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:52:07.106861 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-96399fc5-5b96-42fc-a208-677ec4a366e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:07.113126 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-96399fc5-5b96-42fc-a208-677ec4a366e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:07.115270 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc8736-8e6c-44dc-90eb-4181dc944e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:07.116566 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc8736-8e6c-44dc-90eb-4181dc944e39 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:07.117063 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc8736-8e6c-44dc-90eb-4181dc944e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:07.118795 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc8736-8e6c-44dc-90eb-4181dc944e39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:07.119693 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc8736-8e6c-44dc-90eb-4181dc944e39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:07.121604 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc8736-8e6c-44dc-90eb-4181dc944e39 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:07.123835 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 861/1721] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:52:06 2022] DELETE /identity/v3/projects/189f7f80f724472a9c6ab308fcb18031 => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:07.124945 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96399fc5-5b96-42fc-a208-677ec4a366e5 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:07.130258 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-96399fc5-5b96-42fc-a208-677ec4a366e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:07.144743 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 861/1722] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:07.152117 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc8736-8e6c-44dc-90eb-4181dc944e39 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:07.157226 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-dfdc8736-8e6c-44dc-90eb-4181dc944e39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:07.161419 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96399fc5-5b96-42fc-a208-677ec4a366e5 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:07.179589 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924c639-5543-469b-8281-6b124965d8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:07.180724 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924c639-5543-469b-8281-6b124965d8ea admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:07.181242 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924c639-5543-469b-8281-6b124965d8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:07.181761 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924c639-5543-469b-8281-6b124965d8ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:07.182903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a924c639-5543-469b-8281-6b124965d8ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:07.187655 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a924c639-5543-469b-8281-6b124965d8ea admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:07.190441 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc8736-8e6c-44dc-90eb-4181dc944e39 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:07.203412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc8c34-b36a-4b75-9dc8-e4e099454cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:07.204521 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc8c34-b36a-4b75-9dc8-e4e099454cf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:07.205073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc8c34-b36a-4b75-9dc8-e4e099454cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:07.205712 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc8c34-b36a-4b75-9dc8-e4e099454cf6 admin admin] PATH_INFO: `/v3/users/cb7ae194a9cf46e1802270ba9d90e114` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:07.211898 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc8c34-b36a-4b75-9dc8-e4e099454cf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb7ae194a9cf46e1802270ba9d90e114)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:07.215791 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc8c34-b36a-4b75-9dc8-e4e099454cf6 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:07.219006 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 862/1723] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:07.219006 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924c639-5543-469b-8281-6b124965d8ea admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:07.223338 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-a924c639-5543-469b-8281-6b124965d8ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:07.260929 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924c639-5543-469b-8281-6b124965d8ea admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:07.272132 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8445781-7226-4b6d-a10b-db11e1ec86ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:07.273118 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8445781-7226-4b6d-a10b-db11e1ec86ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:07.274396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8445781-7226-4b6d-a10b-db11e1ec86ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:07.274818 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8445781-7226-4b6d-a10b-db11e1ec86ce admin admin] PATH_INFO: `/v3/projects/b1285de822b54e57a051ae37b9a951d2/users/cdc24602151f4ef4aae12e0ce8be3599/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:07.319385 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8445781-7226-4b6d-a10b-db11e1ec86ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1285de822b54e57a051ae37b9a951d2, user_id=cdc24602151f4ef4aae12e0ce8be3599, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:07.347839 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8445781-7226-4b6d-a10b-db11e1ec86ce admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:07.438743 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 863/1724] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:52:07 2022] PUT /identity/v3/projects/b1285de822b54e57a051ae37b9a951d2/users/cdc24602151f4ef4aae12e0ce8be3599/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:07.439723 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8445781-7226-4b6d-a10b-db11e1ec86ce admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:07.515874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7ffc8c34-b36a-4b75-9dc8-e4e099454cf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb7ae194a9cf46e1802270ba9d90e114'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:07.602148 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7ffc8c34-b36a-4b75-9dc8-e4e099454cf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb7ae194a9cf46e1802270ba9d90e114'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:07.604653 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:07.610098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7ffc8c34-b36a-4b75-9dc8-e4e099454cf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb7ae194a9cf46e1802270ba9d90e114'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:07.617294 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7ffc8c34-b36a-4b75-9dc8-e4e099454cf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb7ae194a9cf46e1802270ba9d90e114'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:07.618924 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7ffc8c34-b36a-4b75-9dc8-e4e099454cf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb7ae194a9cf46e1802270ba9d90e114'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:07.664573 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 862/1725] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:52:07 2022] DELETE /identity/v3/users/cb7ae194a9cf46e1802270ba9d90e114 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:07.679224 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f78d4-b654-4c3f-9a39-fbc59b2913b7 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:07.679224 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f78d4-b654-4c3f-9a39-fbc59b2913b7 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:07.679224 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f78d4-b654-4c3f-9a39-fbc59b2913b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:07.692729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8445781-7226-4b6d-a10b-db11e1ec86ce admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:07.790110 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-e56f78d4-b654-4c3f-9a39-fbc59b2913b7 None None] MFA Rules not processed for user `cdc24602151f4ef4aae12e0ce8be3599`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:07.796263 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-e56f78d4-b654-4c3f-9a39-fbc59b2913b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:07.904522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686f048-ce32-4ff0-aaa5-f1038118a1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:07.904522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686f048-ce32-4ff0-aaa5-f1038118a1da admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:07.904522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686f048-ce32-4ff0-aaa5-f1038118a1da admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:07.904522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686f048-ce32-4ff0-aaa5-f1038118a1da admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:07.904970 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c686f048-ce32-4ff0-aaa5-f1038118a1da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:07.908478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c686f048-ce32-4ff0-aaa5-f1038118a1da admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:07.955203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 864/1726] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 516 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:07.967640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc96cc-142d-4c16-857a-eb4874b15e51 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:07.968228 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc96cc-142d-4c16-857a-eb4874b15e51 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:07.971114 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc96cc-142d-4c16-857a-eb4874b15e51 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:07.977687 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 865/1727] 10.222.0.13 () {54 vars in 912 bytes} [Mon Aug 8 16:52:07 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:52:07.978970 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc96cc-142d-4c16-857a-eb4874b15e51 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:07.991215 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-26fc96cc-142d-4c16-857a-eb4874b15e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:08.005460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 863/1728] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:08.007855 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40da1e-98a4-4d36-a8b4-d277980f9a6e None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:08.012430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40da1e-98a4-4d36-a8b4-d277980f9a6e None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:08.016240 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40da1e-98a4-4d36-a8b4-d277980f9a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:08.041350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc96cc-142d-4c16-857a-eb4874b15e51 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:08.057331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce27ed4-78de-4cc4-9660-3d65696503a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:08.058037 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce27ed4-78de-4cc4-9660-3d65696503a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:08.059034 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce27ed4-78de-4cc4-9660-3d65696503a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:08.060537 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce27ed4-78de-4cc4-9660-3d65696503a7 admin admin] PATH_INFO: `/v3/projects/004a6e287f79459aace6aea4db41020b/users/3ea268ff839a472db8e4d19709916334/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:08.082341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-3d40da1e-98a4-4d36-a8b4-d277980f9a6e None None] MFA Rules not processed for user `9482bf89c1b648c1a91ee00ee5fbba63`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:08.083897 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-3d40da1e-98a4-4d36-a8b4-d277980f9a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:08.118303 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce27ed4-78de-4cc4-9660-3d65696503a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=004a6e287f79459aace6aea4db41020b, user_id=3ea268ff839a472db8e4d19709916334, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:08.125656 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce27ed4-78de-4cc4-9660-3d65696503a7 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:08.255819 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 866/1729] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:52:07 2022] PUT /identity/v3/projects/004a6e287f79459aace6aea4db41020b/users/3ea268ff839a472db8e4d19709916334/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:08.255819 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce27ed4-78de-4cc4-9660-3d65696503a7 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:08.261791 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-cce27ed4-78de-4cc4-9660-3d65696503a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:08.358460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 864/1730] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Aug 8 16:52:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:08.363977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77864d5-013f-4373-8df0-7cb864122945 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:08.365813 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77864d5-013f-4373-8df0-7cb864122945 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:08.367439 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77864d5-013f-4373-8df0-7cb864122945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:08.389824 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:08.513591 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-d77864d5-013f-4373-8df0-7cb864122945 None None] MFA Rules not processed for user `3ea268ff839a472db8e4d19709916334`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:08.514930 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d77864d5-013f-4373-8df0-7cb864122945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:08.629421 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce27ed4-78de-4cc4-9660-3d65696503a7 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:08.644103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ac03f-23bc-4a9c-aa34-f48272df68b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:08.647237 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ac03f-23bc-4a9c-aa34-f48272df68b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:08.652213 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ac03f-23bc-4a9c-aa34-f48272df68b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:08.652213 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ac03f-23bc-4a9c-aa34-f48272df68b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:08.655725 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ac03f-23bc-4a9c-aa34-f48272df68b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:08.668175 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ac03f-23bc-4a9c-aa34-f48272df68b8 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:08.775618 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 865/1731] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:08.778939 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77864d5-013f-4373-8df0-7cb864122945 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:08.783777 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d77864d5-013f-4373-8df0-7cb864122945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:08.849272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 867/1732] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:52:08 2022] POST /identity/v3/projects => generated 365 bytes in 600 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:52:08.849272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ac03f-23bc-4a9c-aa34-f48272df68b8 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:08.858360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-063ac03f-23bc-4a9c-aa34-f48272df68b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:08.989797 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77864d5-013f-4373-8df0-7cb864122945 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:09.063332 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ac03f-23bc-4a9c-aa34-f48272df68b8 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:09.078765 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad141db0-f0a7-4091-a917-f0b0ae717c94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:09.080412 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad141db0-f0a7-4091-a917-f0b0ae717c94 service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:09.080703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad141db0-f0a7-4091-a917-f0b0ae717c94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:09.081279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad141db0-f0a7-4091-a917-f0b0ae717c94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:09.121517 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840d5d2-e53a-480e-9a8f-39e91938cf45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:09.122164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840d5d2-e53a-480e-9a8f-39e91938cf45 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:09.122613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840d5d2-e53a-480e-9a8f-39e91938cf45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:09.122731 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840d5d2-e53a-480e-9a8f-39e91938cf45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:09.142926 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840d5d2-e53a-480e-9a8f-39e91938cf45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:09.142926 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840d5d2-e53a-480e-9a8f-39e91938cf45 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:09.196083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 866/1733] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:09.199799 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840d5d2-e53a-480e-9a8f-39e91938cf45 service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:09.234187 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad141db0-f0a7-4091-a917-f0b0ae717c94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:09.240655 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad141db0-f0a7-4091-a917-f0b0ae717c94 service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:09.256533 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 868/1734] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:09.295486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c66543-2a44-425c-9b85-047b945aca7a None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:09.304424 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c66543-2a44-425c-9b85-047b945aca7a None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:09.305646 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c66543-2a44-425c-9b85-047b945aca7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:09.316608 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:09.351934 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-e9c66543-2a44-425c-9b85-047b945aca7a None None] MFA Rules not processed for user `fdb16e7edbe143e9b96325842896f04e`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:09.353379 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-e9c66543-2a44-425c-9b85-047b945aca7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:09.375510 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840d5d2-e53a-480e-9a8f-39e91938cf45 service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:09.388425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432c1d6-afca-4fb3-be50-9ac3034ada08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:09.389297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432c1d6-afca-4fb3-be50-9ac3034ada08 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:09.389673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432c1d6-afca-4fb3-be50-9ac3034ada08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:09.391499 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432c1d6-afca-4fb3-be50-9ac3034ada08 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:09.395414 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7432c1d6-afca-4fb3-be50-9ac3034ada08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:09.398067 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7432c1d6-afca-4fb3-be50-9ac3034ada08 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:09.496811 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 869/1735] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:52:09.499721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c66543-2a44-425c-9b85-047b945aca7a None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:09.536420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 867/1736] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:52:09 2022] POST /identity/v3/users => generated 289 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:52:09.536420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432c1d6-afca-4fb3-be50-9ac3034ada08 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:09.541600 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-7432c1d6-afca-4fb3-be50-9ac3034ada08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:09.566509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432c1d6-afca-4fb3-be50-9ac3034ada08 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:09.590909 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c66543-2a44-425c-9b85-047b945aca7a None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:09.592436 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84869b-eb2e-46c8-80ee-a13ed8169de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:09.593660 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84869b-eb2e-46c8-80ee-a13ed8169de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:09.593660 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84869b-eb2e-46c8-80ee-a13ed8169de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:09.593815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84869b-eb2e-46c8-80ee-a13ed8169de6 admin admin] PATH_INFO: `/v3/projects/088eb806b7c64badafcc97acce9d56b8` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:09.596790 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84869b-eb2e-46c8-80ee-a13ed8169de6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=088eb806b7c64badafcc97acce9d56b8)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:09.601325 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84869b-eb2e-46c8-80ee-a13ed8169de6 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:09.628322 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528c6ac-4a8a-4048-8d48-e0a96624545a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:09.628993 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528c6ac-4a8a-4048-8d48-e0a96624545a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:09.633264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528c6ac-4a8a-4048-8d48-e0a96624545a admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:09.634125 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528c6ac-4a8a-4048-8d48-e0a96624545a admin admin] PATH_INFO: `/v3/users/3ea268ff839a472db8e4d19709916334` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:09.636785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528c6ac-4a8a-4048-8d48-e0a96624545a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ea268ff839a472db8e4d19709916334)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:09.642847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528c6ac-4a8a-4048-8d48-e0a96624545a admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:09.669729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:09.779620 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-2a84869b-eb2e-46c8-80ee-a13ed8169de6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '088eb806b7c64badafcc97acce9d56b8', 'request_id': 'req-2a84869b-eb2e-46c8-80ee-a13ed8169de6'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:09.793395 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-2a84869b-eb2e-46c8-80ee-a13ed8169de6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '088eb806b7c64badafcc97acce9d56b8', 'request_id': 'req-2a84869b-eb2e-46c8-80ee-a13ed8169de6'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:09.808963 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-2a84869b-eb2e-46c8-80ee-a13ed8169de6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '088eb806b7c64badafcc97acce9d56b8', 'request_id': 'req-2a84869b-eb2e-46c8-80ee-a13ed8169de6'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:09.989243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-2a84869b-eb2e-46c8-80ee-a13ed8169de6 admin admin] The token cache is being invalidate because project 088eb806b7c64badafcc97acce9d56b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:52:09.989243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-2a84869b-eb2e-46c8-80ee-a13ed8169de6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:09.989243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-2a84869b-eb2e-46c8-80ee-a13ed8169de6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:09.989243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 868/1737] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:52:09 2022] DELETE /identity/v3/projects/088eb806b7c64badafcc97acce9d56b8 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:09.991897 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84869b-eb2e-46c8-80ee-a13ed8169de6 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:10.000160 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-2a84869b-eb2e-46c8-80ee-a13ed8169de6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:10.025814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b528c6ac-4a8a-4048-8d48-e0a96624545a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ea268ff839a472db8e4d19709916334'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:10.087734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:10.105430 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b528c6ac-4a8a-4048-8d48-e0a96624545a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ea268ff839a472db8e4d19709916334'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:10.106344 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b528c6ac-4a8a-4048-8d48-e0a96624545a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ea268ff839a472db8e4d19709916334'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:10.129270 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b528c6ac-4a8a-4048-8d48-e0a96624545a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ea268ff839a472db8e4d19709916334'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:10.169889 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b528c6ac-4a8a-4048-8d48-e0a96624545a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ea268ff839a472db8e4d19709916334'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:10.190819 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 870/1738] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:52:09 2022] DELETE /identity/v3/users/3ea268ff839a472db8e4d19709916334 => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:10.192745 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b4576-9b4a-4c45-a9d7-d1f2f1e73be9 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:10.193220 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b4576-9b4a-4c45-a9d7-d1f2f1e73be9 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:10.193780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b4576-9b4a-4c45-a9d7-d1f2f1e73be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:10.281658 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84869b-eb2e-46c8-80ee-a13ed8169de6 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:10.297285 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-f40b4576-9b4a-4c45-a9d7-d1f2f1e73be9 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:10.300458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f40b4576-9b4a-4c45-a9d7-d1f2f1e73be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:10.493481 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efe93a-6e3a-42a8-834e-c815b04a4061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:10.494417 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efe93a-6e3a-42a8-834e-c815b04a4061 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:10.494857 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efe93a-6e3a-42a8-834e-c815b04a4061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:10.495140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efe93a-6e3a-42a8-834e-c815b04a4061 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:10.495802 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81efe93a-6e3a-42a8-834e-c815b04a4061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:10.500612 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81efe93a-6e3a-42a8-834e-c815b04a4061 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:10.528726 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 869/1739] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 539 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:10.530170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38f0e1-8113-40e0-8d44-602844e62265 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:10.532074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38f0e1-8113-40e0-8d44-602844e62265 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:10.532074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38f0e1-8113-40e0-8d44-602844e62265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:10.548641 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 871/1740] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:10.549508 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b4576-9b4a-4c45-a9d7-d1f2f1e73be9 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:10.557335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f40b4576-9b4a-4c45-a9d7-d1f2f1e73be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:10.585529 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b4576-9b4a-4c45-a9d7-d1f2f1e73be9 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:10.603711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-7f38f0e1-8113-40e0-8d44-602844e62265 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:10.607057 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-7f38f0e1-8113-40e0-8d44-602844e62265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:10.612652 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b9c77-b7d8-4781-8043-7c4aa36f64e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:10.614252 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b9c77-b7d8-4781-8043-7c4aa36f64e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:10.614775 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b9c77-b7d8-4781-8043-7c4aa36f64e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:10.618057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b9c77-b7d8-4781-8043-7c4aa36f64e1 admin admin] PATH_INFO: `/v3/projects/3fc1c830b81a42d0ac2e5100b36a23b8/users/966aa03d88c143efbccb15cb25015478/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:10.630853 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 870/1741] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:10.632302 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275ab4b-67d4-44f8-91d7-91d64851f2b8 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:10.632587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275ab4b-67d4-44f8-91d7-91d64851f2b8 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:10.635596 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275ab4b-67d4-44f8-91d7-91d64851f2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:10.656915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b9c77-b7d8-4781-8043-7c4aa36f64e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fc1c830b81a42d0ac2e5100b36a23b8, user_id=966aa03d88c143efbccb15cb25015478, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:10.663592 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b9c77-b7d8-4781-8043-7c4aa36f64e1 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:10.755982 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-7275ab4b-67d4-44f8-91d7-91d64851f2b8 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:10.755982 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-7275ab4b-67d4-44f8-91d7-91d64851f2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:10.803670 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 871/1742] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:10.810262 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275ab4b-67d4-44f8-91d7-91d64851f2b8 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:10.817321 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-7275ab4b-67d4-44f8-91d7-91d64851f2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:10.824857 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 872/1743] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:52:10 2022] PUT /identity/v3/projects/3fc1c830b81a42d0ac2e5100b36a23b8/users/966aa03d88c143efbccb15cb25015478/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:10.824857 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b9c77-b7d8-4781-8043-7c4aa36f64e1 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:10.956106 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:11.014496 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b9c77-b7d8-4781-8043-7c4aa36f64e1 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:11.026689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:11.032452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918bb6b-c463-4d85-b32a-8eed67bc2c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:11.033145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918bb6b-c463-4d85-b32a-8eed67bc2c99 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:11.033428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918bb6b-c463-4d85-b32a-8eed67bc2c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:11.033707 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918bb6b-c463-4d85-b32a-8eed67bc2c99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:11.044610 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918bb6b-c463-4d85-b32a-8eed67bc2c99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:52:11.044909 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918bb6b-c463-4d85-b32a-8eed67bc2c99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:11.054006 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918bb6b-c463-4d85-b32a-8eed67bc2c99 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:11.073479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275ab4b-67d4-44f8-91d7-91d64851f2b8 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:11.092682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 873/1744] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:52:11.093560 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918bb6b-c463-4d85-b32a-8eed67bc2c99 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:11.119993 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918bb6b-c463-4d85-b32a-8eed67bc2c99 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:11.154023 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4ac77-d7a3-48f3-adea-d52adf15e66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:11.157129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4ac77-d7a3-48f3-adea-d52adf15e66f admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:11.157574 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4ac77-d7a3-48f3-adea-d52adf15e66f admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:11.158003 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4ac77-d7a3-48f3-adea-d52adf15e66f admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:11.158865 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b4ac77-d7a3-48f3-adea-d52adf15e66f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:11.166774 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b4ac77-d7a3-48f3-adea-d52adf15e66f admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:11.213950 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 874/1745] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:11.215212 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4ac77-d7a3-48f3-adea-d52adf15e66f admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:11.248404 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4ac77-d7a3-48f3-adea-d52adf15e66f admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:11.262444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84340d-2848-4224-b33f-a2e0d7badbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:11.263853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84340d-2848-4224-b33f-a2e0d7badbf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:11.265407 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84340d-2848-4224-b33f-a2e0d7badbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:11.267099 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84340d-2848-4224-b33f-a2e0d7badbf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:11.268700 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84340d-2848-4224-b33f-a2e0d7badbf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:11.275834 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84340d-2848-4224-b33f-a2e0d7badbf7 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:11.359911 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7368d0-7091-42e1-9ec4-56642cee0ec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:11.361836 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7368d0-7091-42e1-9ec4-56642cee0ec1 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:11.362367 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7368d0-7091-42e1-9ec4-56642cee0ec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:11.362883 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7368d0-7091-42e1-9ec4-56642cee0ec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:11.383162 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7368d0-7091-42e1-9ec4-56642cee0ec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:11.386554 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7368d0-7091-42e1-9ec4-56642cee0ec1 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:11.410134 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 875/1746] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:52:11 2022] POST /identity/v3/projects => generated 357 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:52:11.411306 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84340d-2848-4224-b33f-a2e0d7badbf7 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:11.462060 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 872/1747] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 655 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:11.463329 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7368d0-7091-42e1-9ec4-56642cee0ec1 service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:11.493719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7368d0-7091-42e1-9ec4-56642cee0ec1 service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:11.641555 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a5a71-a261-4a3a-aae6-a85c8c3ea97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:11.642766 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a5a71-a261-4a3a-aae6-a85c8c3ea97f admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:11.643374 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a5a71-a261-4a3a-aae6-a85c8c3ea97f admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:11.643837 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a5a71-a261-4a3a-aae6-a85c8c3ea97f admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:11.645575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a5a71-a261-4a3a-aae6-a85c8c3ea97f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:11.667448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a5a71-a261-4a3a-aae6-a85c8c3ea97f admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:11.674846 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84340d-2848-4224-b33f-a2e0d7badbf7 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:11.686320 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14f6a6-9f3d-4d57-a20a-e598de52a800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:11.688002 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14f6a6-9f3d-4d57-a20a-e598de52a800 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:11.689143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14f6a6-9f3d-4d57-a20a-e598de52a800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:11.690474 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14f6a6-9f3d-4d57-a20a-e598de52a800 admin admin] PATH_INFO: `/v3/projects/3fc1c830b81a42d0ac2e5100b36a23b8/users/966aa03d88c143efbccb15cb25015478/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:11.697368 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14f6a6-9f3d-4d57-a20a-e598de52a800 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fc1c830b81a42d0ac2e5100b36a23b8, user_id=966aa03d88c143efbccb15cb25015478, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:11.703844 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14f6a6-9f3d-4d57-a20a-e598de52a800 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:11.733914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 876/1748] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:52:11 2022] PUT /identity/v3/projects/3fc1c830b81a42d0ac2e5100b36a23b8/users/966aa03d88c143efbccb15cb25015478/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:11.744856 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce14b03-d47c-49b3-8a75-9db2f867fc41 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:11.745652 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce14b03-d47c-49b3-8a75-9db2f867fc41 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:11.745874 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce14b03-d47c-49b3-8a75-9db2f867fc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:11.813703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 873/1749] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:52:11 2022] POST /identity/v3/users => generated 279 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:52:11.817872 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a5a71-a261-4a3a-aae6-a85c8c3ea97f admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:11.824536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-1ce14b03-d47c-49b3-8a75-9db2f867fc41 None None] MFA Rules not processed for user `966aa03d88c143efbccb15cb25015478`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:11.829075 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-1ce14b03-d47c-49b3-8a75-9db2f867fc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:11.859390 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a5a71-a261-4a3a-aae6-a85c8c3ea97f admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:11.900075 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:12.048362 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 877/1750] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:52:12.071442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce14b03-d47c-49b3-8a75-9db2f867fc41 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:12.075730 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-1ce14b03-d47c-49b3-8a75-9db2f867fc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:12.076251 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb5ed7-826b-41f1-b518-dfeaffeb1c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:12.076978 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb5ed7-826b-41f1-b518-dfeaffeb1c08 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:12.077677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb5ed7-826b-41f1-b518-dfeaffeb1c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:12.078076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb5ed7-826b-41f1-b518-dfeaffeb1c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:12.078923 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb5ed7-826b-41f1-b518-dfeaffeb1c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:12.081820 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb5ed7-826b-41f1-b518-dfeaffeb1c08 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:12.100462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 874/1751] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:12.105133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb5ed7-826b-41f1-b518-dfeaffeb1c08 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:12.122429 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb5ed7-826b-41f1-b518-dfeaffeb1c08 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:12.131314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d9625-6152-4255-b869-d8d1ac3e1fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:12.135574 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d9625-6152-4255-b869-d8d1ac3e1fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:12.135973 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d9625-6152-4255-b869-d8d1ac3e1fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:12.136318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d9625-6152-4255-b869-d8d1ac3e1fb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:12.137119 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d9625-6152-4255-b869-d8d1ac3e1fb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:12.139856 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d9625-6152-4255-b869-d8d1ac3e1fb4 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:12.181838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 875/1752] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:12.187270 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d9625-6152-4255-b869-d8d1ac3e1fb4 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:12.212127 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d9625-6152-4255-b869-d8d1ac3e1fb4 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:12.222137 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b87ef-028e-472e-a981-63dffb56222c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:12.222804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b87ef-028e-472e-a981-63dffb56222c admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:12.225407 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b87ef-028e-472e-a981-63dffb56222c admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:12.225682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b87ef-028e-472e-a981-63dffb56222c admin admin] PATH_INFO: `/v3/projects/b6383a76b5b64fafb6caf61928850fc2/users/bbbd725b6b8b4089af57445c1df91337/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:12.296631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19b87ef-028e-472e-a981-63dffb56222c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6383a76b5b64fafb6caf61928850fc2, user_id=bbbd725b6b8b4089af57445c1df91337, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:12.314842 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19b87ef-028e-472e-a981-63dffb56222c admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:12.319001 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce14b03-d47c-49b3-8a75-9db2f867fc41 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:12.342851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89dd8b6-2482-4ed6-9307-4cb034ff5c41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:12.343490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89dd8b6-2482-4ed6-9307-4cb034ff5c41 service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:12.343769 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89dd8b6-2482-4ed6-9307-4cb034ff5c41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:12.344044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89dd8b6-2482-4ed6-9307-4cb034ff5c41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:12.395298 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89dd8b6-2482-4ed6-9307-4cb034ff5c41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:12.396800 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89dd8b6-2482-4ed6-9307-4cb034ff5c41 service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:12.436547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:12.445523 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 876/1753] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:52:12 2022] PUT /identity/v3/projects/b6383a76b5b64fafb6caf61928850fc2/users/bbbd725b6b8b4089af57445c1df91337/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:12.448164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decea76c-533c-44d4-bdf4-27a73a98ac55 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:12.448164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decea76c-533c-44d4-bdf4-27a73a98ac55 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:12.448164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decea76c-533c-44d4-bdf4-27a73a98ac55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:12.454043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 878/1754] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:12 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:12.465047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03c9bf-c0c9-4f5c-9c55-86482556bca3 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:12.465334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03c9bf-c0c9-4f5c-9c55-86482556bca3 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:12.465611 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03c9bf-c0c9-4f5c-9c55-86482556bca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:12.496876 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-decea76c-533c-44d4-bdf4-27a73a98ac55 None None] MFA Rules not processed for user `5a8db3ff64ca44958fd633ba25ee191f`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:12.498752 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-decea76c-533c-44d4-bdf4-27a73a98ac55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:12.528029 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-cd03c9bf-c0c9-4f5c-9c55-86482556bca3 None None] MFA Rules not processed for user `bbbd725b6b8b4089af57445c1df91337`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:12.529455 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-cd03c9bf-c0c9-4f5c-9c55-86482556bca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:12.678453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 879/1755] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:12 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:12.679688 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03c9bf-c0c9-4f5c-9c55-86482556bca3 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:12.749472 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 877/1756] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:12.810999 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decea76c-533c-44d4-bdf4-27a73a98ac55 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:12.828851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03c9bf-c0c9-4f5c-9c55-86482556bca3 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:12.841783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3f75d-75c2-477c-9234-89471b537316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:12.842763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3f75d-75c2-477c-9234-89471b537316 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:12.843192 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3f75d-75c2-477c-9234-89471b537316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:12.843772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3f75d-75c2-477c-9234-89471b537316 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:12.844328 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3f75d-75c2-477c-9234-89471b537316 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:12.846804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3f75d-75c2-477c-9234-89471b537316 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:12.944069 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decea76c-533c-44d4-bdf4-27a73a98ac55 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:12.950117 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3f956-38ad-4652-b983-fe0a60fe5377 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:12.951191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3f956-38ad-4652-b983-fe0a60fe5377 service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:12.951543 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3f956-38ad-4652-b983-fe0a60fe5377 service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:12.951953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3f956-38ad-4652-b983-fe0a60fe5377 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:12.954607 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-45e3f956-38ad-4652-b983-fe0a60fe5377 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:12.995895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 880/1757] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:52:12 2022] POST /identity/v3/projects => generated 357 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:52:12.997728 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634b028-1c52-4ac5-86f2-beea72bbd591 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:12.998241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634b028-1c52-4ac5-86f2-beea72bbd591 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:12.998754 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634b028-1c52-4ac5-86f2-beea72bbd591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:13.134642 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3f956-38ad-4652-b983-fe0a60fe5377 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:13.139410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3f956-38ad-4652-b983-fe0a60fe5377 service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:13.143299 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-b634b028-1c52-4ac5-86f2-beea72bbd591 None None] MFA Rules not processed for user `9646662b0b3048f79ad5d2b9602f10a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:13.147006 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b634b028-1c52-4ac5-86f2-beea72bbd591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:13.171867 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 878/1758] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:52:13.176347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3f956-38ad-4652-b983-fe0a60fe5377 service cinder] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:13.203764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3f956-38ad-4652-b983-fe0a60fe5377 service cinder] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:13.340688 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 881/1759] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Aug 8 16:52:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:13.353343 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cf461-04e2-4ca9-a92d-90d7dd7b57f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:13.354216 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cf461-04e2-4ca9-a92d-90d7dd7b57f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:13.354763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cf461-04e2-4ca9-a92d-90d7dd7b57f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:13.355496 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cf461-04e2-4ca9-a92d-90d7dd7b57f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:13.361192 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cf461-04e2-4ca9-a92d-90d7dd7b57f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:13.361192 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cf461-04e2-4ca9-a92d-90d7dd7b57f4 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:13.400990 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:13.506548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 879/1760] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:52:13 2022] POST /identity/v3/users => generated 285 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:52:13.507671 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634b028-1c52-4ac5-86f2-beea72bbd591 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:13.525982 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634b028-1c52-4ac5-86f2-beea72bbd591 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:13.545060 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4eb21-1237-4e69-9491-d41708f8e08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:13.549373 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4eb21-1237-4e69-9491-d41708f8e08b admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:13.549373 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4eb21-1237-4e69-9491-d41708f8e08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:13.549373 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4eb21-1237-4e69-9491-d41708f8e08b admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:13.549373 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4eb21-1237-4e69-9491-d41708f8e08b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:13.552349 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4eb21-1237-4e69-9491-d41708f8e08b admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:13.557549 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:13.594794 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 882/1761] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:13.599139 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cf461-04e2-4ca9-a92d-90d7dd7b57f4 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:13.615065 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cf461-04e2-4ca9-a92d-90d7dd7b57f4 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:13.623787 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83e467-7c89-4129-ad96-7c28abcb94bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:13.624615 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83e467-7c89-4129-ad96-7c28abcb94bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:13.625107 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83e467-7c89-4129-ad96-7c28abcb94bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:13.625457 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83e467-7c89-4129-ad96-7c28abcb94bb admin admin] PATH_INFO: `/v3/projects/a4f226972b9d470f9370064cff6266d8/users/004b6ad4901345b9a75e7c8d621e084f/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:13.668219 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83e467-7c89-4129-ad96-7c28abcb94bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f226972b9d470f9370064cff6266d8, user_id=004b6ad4901345b9a75e7c8d621e084f, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:13.693502 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83e467-7c89-4129-ad96-7c28abcb94bb admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:13.807007 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 880/1762] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:52:13 2022] PUT /identity/v3/projects/a4f226972b9d470f9370064cff6266d8/users/004b6ad4901345b9a75e7c8d621e084f/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:13.809082 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4eb21-1237-4e69-9491-d41708f8e08b admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:13.822461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4eb21-1237-4e69-9491-d41708f8e08b admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:13.897101 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83e467-7c89-4129-ad96-7c28abcb94bb admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:13.905338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-db83e467-7c89-4129-ad96-7c28abcb94bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:13.990138 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ad983-1b18-4e22-addc-49595e2ef61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:13.990909 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ad983-1b18-4e22-addc-49595e2ef61e admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:13.991364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ad983-1b18-4e22-addc-49595e2ef61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:13.991785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ad983-1b18-4e22-addc-49595e2ef61e admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:13.993452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ad983-1b18-4e22-addc-49595e2ef61e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:13.995511 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ad983-1b18-4e22-addc-49595e2ef61e admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:14.012631 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 883/1763] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:14.016596 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ad983-1b18-4e22-addc-49595e2ef61e admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:14.037904 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ad983-1b18-4e22-addc-49595e2ef61e admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:14.050515 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206a2cc-0cd3-4478-b0d7-b234efcd2436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:14.051531 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206a2cc-0cd3-4478-b0d7-b234efcd2436 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:14.052115 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206a2cc-0cd3-4478-b0d7-b234efcd2436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:14.052616 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206a2cc-0cd3-4478-b0d7-b234efcd2436 admin admin] PATH_INFO: `/v3/projects/a4f226972b9d470f9370064cff6266d8/users/004b6ad4901345b9a75e7c8d621e084f/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:14.056877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206a2cc-0cd3-4478-b0d7-b234efcd2436 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f226972b9d470f9370064cff6266d8, user_id=004b6ad4901345b9a75e7c8d621e084f, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:14.062734 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206a2cc-0cd3-4478-b0d7-b234efcd2436 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:14.095584 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 884/1764] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:52:14 2022] PUT /identity/v3/projects/a4f226972b9d470f9370064cff6266d8/users/004b6ad4901345b9a75e7c8d621e084f/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:14.096962 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206a2cc-0cd3-4478-b0d7-b234efcd2436 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:14.105342 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-c206a2cc-0cd3-4478-b0d7-b234efcd2436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:14.318661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206a2cc-0cd3-4478-b0d7-b234efcd2436 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:14.331636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b9431-407e-43a3-955e-09db781077bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:14.332269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b9431-407e-43a3-955e-09db781077bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:14.332581 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b9431-407e-43a3-955e-09db781077bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:14.332810 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b9431-407e-43a3-955e-09db781077bc admin admin] PATH_INFO: `/v3/projects/004a6e287f79459aace6aea4db41020b` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:14.338011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b9431-407e-43a3-955e-09db781077bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=004a6e287f79459aace6aea4db41020b)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:14.340378 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b9431-407e-43a3-955e-09db781077bc admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:14.401154 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83e467-7c89-4129-ad96-7c28abcb94bb admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:14.526708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f3845-36d7-4174-bd30-00ff289a2653 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c606decb832444e38a194b98f8d15f8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:14.529456 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f3845-36d7-4174-bd30-00ff289a2653 service swift] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:14.529922 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f3845-36d7-4174-bd30-00ff289a2653 service swift] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:14.530409 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f3845-36d7-4174-bd30-00ff289a2653 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:14.562640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-689b9431-407e-43a3-955e-09db781077bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '004a6e287f79459aace6aea4db41020b', 'request_id': 'req-689b9431-407e-43a3-955e-09db781077bc'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:14.649207 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-689b9431-407e-43a3-955e-09db781077bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '004a6e287f79459aace6aea4db41020b', 'request_id': 'req-689b9431-407e-43a3-955e-09db781077bc'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:14.662068 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-689b9431-407e-43a3-955e-09db781077bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '004a6e287f79459aace6aea4db41020b', 'request_id': 'req-689b9431-407e-43a3-955e-09db781077bc'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:14.685972 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f3845-36d7-4174-bd30-00ff289a2653 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:14.690104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f3845-36d7-4174-bd30-00ff289a2653 service swift] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:14.692559 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-689b9431-407e-43a3-955e-09db781077bc admin admin] The token cache is being invalidate because project 004a6e287f79459aace6aea4db41020b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:52:14.698789 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-689b9431-407e-43a3-955e-09db781077bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:14.700109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-689b9431-407e-43a3-955e-09db781077bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:14.709068 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 885/1765] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:52:14 2022] DELETE /identity/v3/projects/004a6e287f79459aace6aea4db41020b => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:14.717751 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1ed05-c1cd-4b16-a0c7-305c2fa4d305 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:14.717751 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1ed05-c1cd-4b16-a0c7-305c2fa4d305 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:14.717751 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1ed05-c1cd-4b16-a0c7-305c2fa4d305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:14.724762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 881/1766] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Aug 8 16:52:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 828 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 16:52:14.728390 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bce10-5307-4125-82dc-2ef5b9586314 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:14.728675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bce10-5307-4125-82dc-2ef5b9586314 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:14.730678 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bce10-5307-4125-82dc-2ef5b9586314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:14.790997 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-397bce10-5307-4125-82dc-2ef5b9586314 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:14.795166 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-397bce10-5307-4125-82dc-2ef5b9586314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:14.796971 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-36a1ed05-c1cd-4b16-a0c7-305c2fa4d305 None None] MFA Rules not processed for user `004b6ad4901345b9a75e7c8d621e084f`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:14.800835 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-36a1ed05-c1cd-4b16-a0c7-305c2fa4d305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:14.808716 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:15.001515 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:15.055798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 882/1767] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:15.066813 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef989a4f-3cdf-4964-8658-88dbce634504 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:15.067341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef989a4f-3cdf-4964-8658-88dbce634504 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:15.071914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef989a4f-3cdf-4964-8658-88dbce634504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:15.137425 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 886/1768] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:14 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:15.228350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1ed05-c1cd-4b16-a0c7-305c2fa4d305 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:15.229510 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-ef989a4f-3cdf-4964-8658-88dbce634504 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:15.229510 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ef989a4f-3cdf-4964-8658-88dbce634504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:15.229870 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-36a1ed05-c1cd-4b16-a0c7-305c2fa4d305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:15.229961 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 883/1769] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:15.229961 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ea051-7339-4bf7-a47e-e59bc6e22ab2 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:15.229961 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ea051-7339-4bf7-a47e-e59bc6e22ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:15.229961 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ea051-7339-4bf7-a47e-e59bc6e22ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:15.235837 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-ee2ea051-7339-4bf7-a47e-e59bc6e22ab2 None None] MFA Rules not processed for user `9482bf89c1b648c1a91ee00ee5fbba63`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:15.237148 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ee2ea051-7339-4bf7-a47e-e59bc6e22ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:15.345565 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1ed05-c1cd-4b16-a0c7-305c2fa4d305 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:15.359314 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2cf1b-13e2-46ed-8ca3-d9c467ec5cbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:15.360495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2cf1b-13e2-46ed-8ca3-d9c467ec5cbd service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:15.360926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2cf1b-13e2-46ed-8ca3-d9c467ec5cbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:15.361354 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2cf1b-13e2-46ed-8ca3-d9c467ec5cbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:15.405869 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2cf1b-13e2-46ed-8ca3-d9c467ec5cbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:15.413649 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2cf1b-13e2-46ed-8ca3-d9c467ec5cbd service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:15.453150 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 887/1770] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:15 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:15.460426 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2cf1b-13e2-46ed-8ca3-d9c467ec5cbd service cinder] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:15.487488 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2cf1b-13e2-46ed-8ca3-d9c467ec5cbd service cinder] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:15.502595 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb04f8e-6e9d-4661-b400-9833f997fb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:15.503467 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb04f8e-6e9d-4661-b400-9833f997fb31 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:15.503805 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb04f8e-6e9d-4661-b400-9833f997fb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:15.504072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb04f8e-6e9d-4661-b400-9833f997fb31 admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:15.504764 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb04f8e-6e9d-4661-b400-9833f997fb31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:52:15.505058 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb04f8e-6e9d-4661-b400-9833f997fb31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:15.510791 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb04f8e-6e9d-4661-b400-9833f997fb31 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:15.531673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 884/1771] 10.222.0.23 () {58 vars in 1015 bytes} [Mon Aug 8 16:52:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:15.557221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 888/1772] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:52:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:52:15.558758 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ea051-7339-4bf7-a47e-e59bc6e22ab2 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:15.588280 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ea051-7339-4bf7-a47e-e59bc6e22ab2 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:15.595469 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3062087-25df-4e96-be00-9e2f64207efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:15.596290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3062087-25df-4e96-be00-9e2f64207efe admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:15.596760 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3062087-25df-4e96-be00-9e2f64207efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:15.597368 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3062087-25df-4e96-be00-9e2f64207efe admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:15.598897 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3062087-25df-4e96-be00-9e2f64207efe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:15.630727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3062087-25df-4e96-be00-9e2f64207efe admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:15.747151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb04f8e-6e9d-4661-b400-9833f997fb31 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:15.747151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-cbb04f8e-6e9d-4661-b400-9833f997fb31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:15.771187 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb04f8e-6e9d-4661-b400-9833f997fb31 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:15.783165 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3dec2-463c-4cec-984c-6c7491795333 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:15.783814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3dec2-463c-4cec-984c-6c7491795333 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:15.784093 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3dec2-463c-4cec-984c-6c7491795333 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:15.784363 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3dec2-463c-4cec-984c-6c7491795333 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:15.807499 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c3dec2-463c-4cec-984c-6c7491795333 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:15.811846 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c3dec2-463c-4cec-984c-6c7491795333 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:15.852900 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 889/1773] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:52:15.877800 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 885/1774] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:52:15 2022] POST /identity/v3/projects => generated 361 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:52:15.879149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3dec2-463c-4cec-984c-6c7491795333 service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:15.895074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3062087-25df-4e96-be00-9e2f64207efe admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:15.895291 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3dec2-463c-4cec-984c-6c7491795333 service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:15.972616 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:16.089412 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7a767-2997-4389-a590-f0da13c1e389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:16.090482 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7a767-2997-4389-a590-f0da13c1e389 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:16.091035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7a767-2997-4389-a590-f0da13c1e389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:16.091879 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7a767-2997-4389-a590-f0da13c1e389 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:16.092753 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7a767-2997-4389-a590-f0da13c1e389 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:16.095719 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7a767-2997-4389-a590-f0da13c1e389 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:16.100375 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3062087-25df-4e96-be00-9e2f64207efe admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:16.109738 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a28fc-9ba6-4e6d-858e-41f2415dd977 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:16.110496 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a28fc-9ba6-4e6d-858e-41f2415dd977 service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:16.110961 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a28fc-9ba6-4e6d-858e-41f2415dd977 service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:16.111545 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a28fc-9ba6-4e6d-858e-41f2415dd977 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:16.115642 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-a98a28fc-9ba6-4e6d-858e-41f2415dd977 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:16.283039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 890/1775] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:52:15 2022] POST /identity/v3/users => generated 281 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:52:16.283039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7a767-2997-4389-a590-f0da13c1e389 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:16.309833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7a767-2997-4389-a590-f0da13c1e389 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:16.323471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d333a4-5540-4b3e-bb3d-9a80fe8ca941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:16.324100 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d333a4-5540-4b3e-bb3d-9a80fe8ca941 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:16.324433 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d333a4-5540-4b3e-bb3d-9a80fe8ca941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:16.324802 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d333a4-5540-4b3e-bb3d-9a80fe8ca941 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:16.325497 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d333a4-5540-4b3e-bb3d-9a80fe8ca941 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:16.330283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d333a4-5540-4b3e-bb3d-9a80fe8ca941 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:16.346107 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a28fc-9ba6-4e6d-858e-41f2415dd977 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:16.347617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a28fc-9ba6-4e6d-858e-41f2415dd977 service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:16.371935 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 891/1776] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:52:16.373669 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 886/1777] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:16.389478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d333a4-5540-4b3e-bb3d-9a80fe8ca941 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:16.389478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:16.437050 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d333a4-5540-4b3e-bb3d-9a80fe8ca941 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:16.465246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dae605-6d95-4174-a3f5-700865038377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:16.472196 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dae605-6d95-4174-a3f5-700865038377 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:16.479880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dae605-6d95-4174-a3f5-700865038377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:16.480406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dae605-6d95-4174-a3f5-700865038377 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:16.481260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dae605-6d95-4174-a3f5-700865038377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:16.483271 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dae605-6d95-4174-a3f5-700865038377 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:16.549264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 892/1778] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:16.559685 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a28fc-9ba6-4e6d-858e-41f2415dd977 service cinder] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:16.578222 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a28fc-9ba6-4e6d-858e-41f2415dd977 service cinder] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:16.616205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d042505-bbc7-4bd4-ae83-7c4c1a6f40ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:16.616934 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d042505-bbc7-4bd4-ae83-7c4c1a6f40ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:16.617344 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d042505-bbc7-4bd4-ae83-7c4c1a6f40ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:16.617736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d042505-bbc7-4bd4-ae83-7c4c1a6f40ef admin admin] PATH_INFO: `/v3/projects/b17ad95eab8641ad99506a550b60d64d/users/62590919f6194bbeadbbd46985905360/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:16.713953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d042505-bbc7-4bd4-ae83-7c4c1a6f40ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=b17ad95eab8641ad99506a550b60d64d, user_id=62590919f6194bbeadbbd46985905360, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:16.716423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d042505-bbc7-4bd4-ae83-7c4c1a6f40ef admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:16.794113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 887/1779] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:52:16 2022] PUT /identity/v3/projects/b17ad95eab8641ad99506a550b60d64d/users/62590919f6194bbeadbbd46985905360/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:16.805739 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389bc17-f03f-4d75-be04-6bc9aca95bb6 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:16.806504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389bc17-f03f-4d75-be04-6bc9aca95bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:16.806504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389bc17-f03f-4d75-be04-6bc9aca95bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:16.956188 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-a389bc17-f03f-4d75-be04-6bc9aca95bb6 None None] MFA Rules not processed for user `62590919f6194bbeadbbd46985905360`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:16.956188 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-a389bc17-f03f-4d75-be04-6bc9aca95bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:17.265317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 893/1780] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:17.308431 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d042505-bbc7-4bd4-ae83-7c4c1a6f40ef admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:17.446622 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d042505-bbc7-4bd4-ae83-7c4c1a6f40ef admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:17.452927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf1f305-7a0f-423e-81c5-2ead6d10385b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:17.453554 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1f305-7a0f-423e-81c5-2ead6d10385b service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:17.453876 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1f305-7a0f-423e-81c5-2ead6d10385b service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:17.454553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1f305-7a0f-423e-81c5-2ead6d10385b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:17.474708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf1f305-7a0f-423e-81c5-2ead6d10385b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:17.476246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf1f305-7a0f-423e-81c5-2ead6d10385b service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:17.498601 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 888/1781] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:17.519977 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389bc17-f03f-4d75-be04-6bc9aca95bb6 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:17.533277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:17.723725 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389bc17-f03f-4d75-be04-6bc9aca95bb6 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:17.854484 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb77a16-3516-4e6d-9ff3-f5185f0d2b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:17.854484 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb77a16-3516-4e6d-9ff3-f5185f0d2b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:17.854484 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb77a16-3516-4e6d-9ff3-f5185f0d2b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:17.854484 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb77a16-3516-4e6d-9ff3-f5185f0d2b06 admin admin] PATH_INFO: `/v3/users/62590919f6194bbeadbbd46985905360` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:17.855617 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb77a16-3516-4e6d-9ff3-f5185f0d2b06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62590919f6194bbeadbbd46985905360)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:17.855617 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb77a16-3516-4e6d-9ff3-f5185f0d2b06 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:18.189925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:18.218844 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-cbb77a16-3516-4e6d-9ff3-f5185f0d2b06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62590919f6194bbeadbbd46985905360'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:18.316502 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-cbb77a16-3516-4e6d-9ff3-f5185f0d2b06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62590919f6194bbeadbbd46985905360'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:18.317573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-cbb77a16-3516-4e6d-9ff3-f5185f0d2b06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62590919f6194bbeadbbd46985905360'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:18.356944 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-cbb77a16-3516-4e6d-9ff3-f5185f0d2b06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62590919f6194bbeadbbd46985905360'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:18.397797 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-cbb77a16-3516-4e6d-9ff3-f5185f0d2b06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62590919f6194bbeadbbd46985905360'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:18.430469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 894/1782] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:52:17 2022] DELETE /identity/v3/users/62590919f6194bbeadbbd46985905360 => generated 0 bytes in 907 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:18.432882 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00180c-083d-41bf-ad5e-d8868a904455 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:18.434434 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00180c-083d-41bf-ad5e-d8868a904455 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:18.435108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00180c-083d-41bf-ad5e-d8868a904455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:18.480610 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-6a00180c-083d-41bf-ad5e-d8868a904455 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:18.483314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-6a00180c-083d-41bf-ad5e-d8868a904455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:18.646587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 889/1783] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:18.909669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb77a16-3516-4e6d-9ff3-f5185f0d2b06 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:18.911861 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-cbb77a16-3516-4e6d-9ff3-f5185f0d2b06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:19.103547 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb77a16-3516-4e6d-9ff3-f5185f0d2b06 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:19.115395 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a256f856-63d8-4c0d-9dc4-75c7674f59a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:19.116309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256f856-63d8-4c0d-9dc4-75c7674f59a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:19.117517 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256f856-63d8-4c0d-9dc4-75c7674f59a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:19.118022 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256f856-63d8-4c0d-9dc4-75c7674f59a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:19.148945 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a256f856-63d8-4c0d-9dc4-75c7674f59a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:19.152415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a256f856-63d8-4c0d-9dc4-75c7674f59a6 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:19.181062 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 895/1784] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:19.437824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:19.911753 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00180c-083d-41bf-ad5e-d8868a904455 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:19.917024 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-6a00180c-083d-41bf-ad5e-d8868a904455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:19.918143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbf200-7389-4588-acf0-277120259c62 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:19.919050 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbf200-7389-4588-acf0-277120259c62 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:19.919483 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbf200-7389-4588-acf0-277120259c62 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:19.921616 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 896/1785] 10.222.0.13 () {54 vars in 921 bytes} [Mon Aug 8 16:52:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:52:19.940457 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2a32a-bf30-4e3c-b333-6d9e5eb482bb None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:19.940982 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2a32a-bf30-4e3c-b333-6d9e5eb482bb None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:19.941432 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2a32a-bf30-4e3c-b333-6d9e5eb482bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:20.004812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-50e2a32a-bf30-4e3c-b333-6d9e5eb482bb None None] MFA Rules not processed for user `9daa7489bced467984d39e8b8ac49d13`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:20.006864 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-50e2a32a-bf30-4e3c-b333-6d9e5eb482bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:20.069956 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00180c-083d-41bf-ad5e-d8868a904455 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:20.130278 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7532056-f895-4864-95b6-8e64b5fe9922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:20.144676 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7532056-f895-4864-95b6-8e64b5fe9922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:20.145496 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7532056-f895-4864-95b6-8e64b5fe9922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:20.147661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7532056-f895-4864-95b6-8e64b5fe9922 admin admin] PATH_INFO: `/v3/users/cdc24602151f4ef4aae12e0ce8be3599` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:20.168046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7532056-f895-4864-95b6-8e64b5fe9922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdc24602151f4ef4aae12e0ce8be3599)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:20.175830 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7532056-f895-4864-95b6-8e64b5fe9922 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:20.524366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 897/1786] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Aug 8 16:52:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 585 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:52:20.740928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e7532056-f895-4864-95b6-8e64b5fe9922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdc24602151f4ef4aae12e0ce8be3599'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:20.847515 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e7532056-f895-4864-95b6-8e64b5fe9922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdc24602151f4ef4aae12e0ce8be3599'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:20.848070 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e7532056-f895-4864-95b6-8e64b5fe9922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdc24602151f4ef4aae12e0ce8be3599'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:20.861708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e7532056-f895-4864-95b6-8e64b5fe9922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdc24602151f4ef4aae12e0ce8be3599'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:20.862450 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e7532056-f895-4864-95b6-8e64b5fe9922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdc24602151f4ef4aae12e0ce8be3599'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:20.926936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 890/1787] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:52:19 2022] DELETE /identity/v3/users/cdc24602151f4ef4aae12e0ce8be3599 => generated 0 bytes in 1016 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:20.933222 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2ff53-c1c9-446e-a27b-5c72f199dbab None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:20.934292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2ff53-c1c9-446e-a27b-5c72f199dbab None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:20.934853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2ff53-c1c9-446e-a27b-5c72f199dbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:21.004136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-47f2ff53-c1c9-446e-a27b-5c72f199dbab None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:21.010443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-47f2ff53-c1c9-446e-a27b-5c72f199dbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:21.255906 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 898/1788] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:21.352291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7532056-f895-4864-95b6-8e64b5fe9922 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:21.353934 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-e7532056-f895-4864-95b6-8e64b5fe9922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:21.527338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7532056-f895-4864-95b6-8e64b5fe9922 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:21.564121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eaa4d2-3f29-4f86-bfa7-b407084b3d79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:21.566205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eaa4d2-3f29-4f86-bfa7-b407084b3d79 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:21.566205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eaa4d2-3f29-4f86-bfa7-b407084b3d79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:21.577638 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eaa4d2-3f29-4f86-bfa7-b407084b3d79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:21.593560 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eaa4d2-3f29-4f86-bfa7-b407084b3d79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:21.594951 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eaa4d2-3f29-4f86-bfa7-b407084b3d79 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:21.626293 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 891/1789] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:21.969175 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2ff53-c1c9-446e-a27b-5c72f199dbab None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:21.970846 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-47f2ff53-c1c9-446e-a27b-5c72f199dbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:22.109525 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96a9f8-ce64-40d0-aca0-c623478335e3 None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:22.110890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96a9f8-ce64-40d0-aca0-c623478335e3 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:22.111616 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96a9f8-ce64-40d0-aca0-c623478335e3 None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:22.116126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 892/1790] 10.222.0.13 () {54 vars in 921 bytes} [Mon Aug 8 16:52:22 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:52:22.203525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2ff53-c1c9-446e-a27b-5c72f199dbab None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:22.214618 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96a9f8-ce64-40d0-aca0-c623478335e3 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:22.218952 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-da96a9f8-ce64-40d0-aca0-c623478335e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:22.221505 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0cf04-7424-428b-aa38-b0b8508d251e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:22.222471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0cf04-7424-428b-aa38-b0b8508d251e service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:22.223019 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0cf04-7424-428b-aa38-b0b8508d251e service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:22.223502 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0cf04-7424-428b-aa38-b0b8508d251e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:22.226249 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-71c0cf04-7424-428b-aa38-b0b8508d251e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:22.244589 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0cf04-7424-428b-aa38-b0b8508d251e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:22.249015 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0cf04-7424-428b-aa38-b0b8508d251e service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:22.285211 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 899/1791] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:22.287428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abedb6-56aa-4b6c-afc3-3766a1dc1c0a None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:22.287986 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abedb6-56aa-4b6c-afc3-3766a1dc1c0a None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:22.288655 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abedb6-56aa-4b6c-afc3-3766a1dc1c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:22.355194 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-84abedb6-56aa-4b6c-afc3-3766a1dc1c0a None None] MFA Rules not processed for user `9daa7489bced467984d39e8b8ac49d13`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:22.357197 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-84abedb6-56aa-4b6c-afc3-3766a1dc1c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:22.381501 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96a9f8-ce64-40d0-aca0-c623478335e3 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:22.395326 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 900/1792] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Aug 8 16:52:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:22.423955 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe586a61-a16e-490e-b4ce-7e284bd017cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:22.429639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe586a61-a16e-490e-b4ce-7e284bd017cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:22.430414 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe586a61-a16e-490e-b4ce-7e284bd017cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:22.431564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe586a61-a16e-490e-b4ce-7e284bd017cc admin admin] PATH_INFO: `/v3/projects/b17ad95eab8641ad99506a550b60d64d` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:22.437876 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe586a61-a16e-490e-b4ce-7e284bd017cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b17ad95eab8641ad99506a550b60d64d)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:22.450285 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe586a61-a16e-490e-b4ce-7e284bd017cc admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:22.636344 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:22.681588 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-fe586a61-a16e-490e-b4ce-7e284bd017cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b17ad95eab8641ad99506a550b60d64d', 'request_id': 'req-fe586a61-a16e-490e-b4ce-7e284bd017cc'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:22.706003 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-fe586a61-a16e-490e-b4ce-7e284bd017cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b17ad95eab8641ad99506a550b60d64d', 'request_id': 'req-fe586a61-a16e-490e-b4ce-7e284bd017cc'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:22.731795 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-fe586a61-a16e-490e-b4ce-7e284bd017cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b17ad95eab8641ad99506a550b60d64d', 'request_id': 'req-fe586a61-a16e-490e-b4ce-7e284bd017cc'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:22.934618 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-fe586a61-a16e-490e-b4ce-7e284bd017cc admin admin] The token cache is being invalidate because project b17ad95eab8641ad99506a550b60d64d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:52:22.934811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-fe586a61-a16e-490e-b4ce-7e284bd017cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:22.935743 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-fe586a61-a16e-490e-b4ce-7e284bd017cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:22.943081 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fdf72-cda6-4410-a8f9-59d5f01da596 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:22.943844 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fdf72-cda6-4410-a8f9-59d5f01da596 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:22.944273 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fdf72-cda6-4410-a8f9-59d5f01da596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:22.948442 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 893/1793] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:52:22 2022] DELETE /identity/v3/projects/b17ad95eab8641ad99506a550b60d64d => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:23.019489 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-1a1fdf72-cda6-4410-a8f9-59d5f01da596 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:23.021912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-1a1fdf72-cda6-4410-a8f9-59d5f01da596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:23.173709 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:23.203476 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 901/1794] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:23.209625 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44e7d5-a9f7-4182-afd3-9ad4ae984660 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:23.211952 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44e7d5-a9f7-4182-afd3-9ad4ae984660 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:23.212486 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44e7d5-a9f7-4182-afd3-9ad4ae984660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:23.325826 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-1b44e7d5-a9f7-4182-afd3-9ad4ae984660 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:23.339236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-1b44e7d5-a9f7-4182-afd3-9ad4ae984660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:23.360668 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 894/1795] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:23.365390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fdf72-cda6-4410-a8f9-59d5f01da596 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:23.394878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fdf72-cda6-4410-a8f9-59d5f01da596 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:23.405151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee86bf3-cc9d-478d-898a-e2c6cf62f0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:23.406249 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee86bf3-cc9d-478d-898a-e2c6cf62f0d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:23.406799 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee86bf3-cc9d-478d-898a-e2c6cf62f0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:23.407525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee86bf3-cc9d-478d-898a-e2c6cf62f0d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:23.408988 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee86bf3-cc9d-478d-898a-e2c6cf62f0d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:52:23.409764 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee86bf3-cc9d-478d-898a-e2c6cf62f0d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:23.426183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44e7d5-a9f7-4182-afd3-9ad4ae984660 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:23.440542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee86bf3-cc9d-478d-898a-e2c6cf62f0d8 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:23.440929 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-1b44e7d5-a9f7-4182-afd3-9ad4ae984660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:23.475665 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 902/1796] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:52:23.495979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee86bf3-cc9d-478d-898a-e2c6cf62f0d8 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:23.536174 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee86bf3-cc9d-478d-898a-e2c6cf62f0d8 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:23.567518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c110138-6f58-49fb-9538-991e132ea18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:23.572361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c110138-6f58-49fb-9538-991e132ea18b admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:23.573483 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c110138-6f58-49fb-9538-991e132ea18b admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:23.574251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c110138-6f58-49fb-9538-991e132ea18b admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:23.577011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c110138-6f58-49fb-9538-991e132ea18b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:23.591200 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c110138-6f58-49fb-9538-991e132ea18b admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:23.647377 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44e7d5-a9f7-4182-afd3-9ad4ae984660 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:23.663046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc3da1-625e-4c21-bfbc-1d9c7493b4e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:23.663684 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc3da1-625e-4c21-bfbc-1d9c7493b4e5 service nova] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:23.664333 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc3da1-625e-4c21-bfbc-1d9c7493b4e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:23.664684 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc3da1-625e-4c21-bfbc-1d9c7493b4e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:23.671746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-97fc3da1-625e-4c21-bfbc-1d9c7493b4e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:23.704757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc3da1-625e-4c21-bfbc-1d9c7493b4e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:23.708285 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc3da1-625e-4c21-bfbc-1d9c7493b4e5 service nova] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:23.738980 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 895/1797] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:23.838951 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 903/1798] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:52:23 2022] POST /identity/v3/projects => generated 385 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:52:23.841421 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc3da1-625e-4c21-bfbc-1d9c7493b4e5 service nova] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:23.886240 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc3da1-625e-4c21-bfbc-1d9c7493b4e5 service nova] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:24.145267 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3d356-59f2-4455-8885-c71afc8c11d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:24.146173 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3d356-59f2-4455-8885-c71afc8c11d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:24.146526 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3d356-59f2-4455-8885-c71afc8c11d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:24.146837 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3d356-59f2-4455-8885-c71afc8c11d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:24.147500 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3d356-59f2-4455-8885-c71afc8c11d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:24.151100 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3d356-59f2-4455-8885-c71afc8c11d5 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:24.630373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 896/1799] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:52:23 2022] POST /identity/v3/users => generated 293 bytes in 789 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:52:24.635669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c110138-6f58-49fb-9538-991e132ea18b admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:24.661630 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c110138-6f58-49fb-9538-991e132ea18b admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:24.683303 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb060aa2-4f35-4927-8fbe-bf55ed61b5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:24.687856 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb060aa2-4f35-4927-8fbe-bf55ed61b5da admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:24.687856 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb060aa2-4f35-4927-8fbe-bf55ed61b5da admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:24.687856 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb060aa2-4f35-4927-8fbe-bf55ed61b5da admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:24.687856 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb060aa2-4f35-4927-8fbe-bf55ed61b5da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:24.698065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb060aa2-4f35-4927-8fbe-bf55ed61b5da admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:24.736782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 904/1800] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:24.736950 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3d356-59f2-4455-8885-c71afc8c11d5 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:24.753503 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3d356-59f2-4455-8885-c71afc8c11d5 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:24.795397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d446d-337c-4853-8af5-46553279b612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:24.798547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d446d-337c-4853-8af5-46553279b612 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:24.798987 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d446d-337c-4853-8af5-46553279b612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:24.799386 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d446d-337c-4853-8af5-46553279b612 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:24.800154 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d446d-337c-4853-8af5-46553279b612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:24.801954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d446d-337c-4853-8af5-46553279b612 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:24.829260 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 897/1801] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:24.830845 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb060aa2-4f35-4927-8fbe-bf55ed61b5da admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:24.850390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb060aa2-4f35-4927-8fbe-bf55ed61b5da admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:24.868559 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12054805-da0c-4f72-a05c-23c8a528d4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:24.869662 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12054805-da0c-4f72-a05c-23c8a528d4f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:24.870179 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12054805-da0c-4f72-a05c-23c8a528d4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:24.870705 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12054805-da0c-4f72-a05c-23c8a528d4f0 admin admin] PATH_INFO: `/v3/projects/5e4d79d7beca4f9292e13c51d9633317/users/5f0008eec3394de093222264494ea4f0/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:24.917610 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12054805-da0c-4f72-a05c-23c8a528d4f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e4d79d7beca4f9292e13c51d9633317, user_id=5f0008eec3394de093222264494ea4f0, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:24.925860 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12054805-da0c-4f72-a05c-23c8a528d4f0 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:25.109461 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d446d-337c-4853-8af5-46553279b612 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:25.111392 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-533d446d-337c-4853-8af5-46553279b612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:25.138193 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d446d-337c-4853-8af5-46553279b612 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:25.154743 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c07819-a15a-4b77-94b4-7f4c97e77e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:25.156701 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c07819-a15a-4b77-94b4-7f4c97e77e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:25.157254 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c07819-a15a-4b77-94b4-7f4c97e77e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:25.157765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c07819-a15a-4b77-94b4-7f4c97e77e06 admin admin] PATH_INFO: `/v3/projects/b1285de822b54e57a051ae37b9a951d2` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:25.159417 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c07819-a15a-4b77-94b4-7f4c97e77e06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1285de822b54e57a051ae37b9a951d2)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:25.161818 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c07819-a15a-4b77-94b4-7f4c97e77e06 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:25.179300 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 905/1802] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:52:24 2022] PUT /identity/v3/projects/5e4d79d7beca4f9292e13c51d9633317/users/5f0008eec3394de093222264494ea4f0/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:25.182589 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54880cd1-cbc7-4a47-b9ce-b4dcb6149ae8 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:25.188273 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54880cd1-cbc7-4a47-b9ce-b4dcb6149ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:25.189501 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54880cd1-cbc7-4a47-b9ce-b4dcb6149ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:25.261437 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-54880cd1-cbc7-4a47-b9ce-b4dcb6149ae8 None None] MFA Rules not processed for user `5f0008eec3394de093222264494ea4f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:25.261437 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-54880cd1-cbc7-4a47-b9ce-b4dcb6149ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:25.405826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 906/1803] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:25.413104 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54880cd1-cbc7-4a47-b9ce-b4dcb6149ae8 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:25.509832 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f4c07819-a15a-4b77-94b4-7f4c97e77e06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1285de822b54e57a051ae37b9a951d2', 'request_id': 'req-f4c07819-a15a-4b77-94b4-7f4c97e77e06'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:25.526302 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f4c07819-a15a-4b77-94b4-7f4c97e77e06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1285de822b54e57a051ae37b9a951d2', 'request_id': 'req-f4c07819-a15a-4b77-94b4-7f4c97e77e06'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:25.557571 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f4c07819-a15a-4b77-94b4-7f4c97e77e06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1285de822b54e57a051ae37b9a951d2', 'request_id': 'req-f4c07819-a15a-4b77-94b4-7f4c97e77e06'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:25.597708 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54880cd1-cbc7-4a47-b9ce-b4dcb6149ae8 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:25.717539 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f4c07819-a15a-4b77-94b4-7f4c97e77e06 admin admin] The token cache is being invalidate because project b1285de822b54e57a051ae37b9a951d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:52:25.718233 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f4c07819-a15a-4b77-94b4-7f4c97e77e06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:25.719014 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f4c07819-a15a-4b77-94b4-7f4c97e77e06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:25.721851 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 898/1804] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:52:25 2022] DELETE /identity/v3/projects/b1285de822b54e57a051ae37b9a951d2 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:52:25.734287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c07819-a15a-4b77-94b4-7f4c97e77e06 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:25.743826 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-f4c07819-a15a-4b77-94b4-7f4c97e77e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:25.771714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103bed5-0334-43d8-ad9a-ece7fe29afb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:25.775402 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103bed5-0334-43d8-ad9a-ece7fe29afb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:25.775402 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103bed5-0334-43d8-ad9a-ece7fe29afb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:25.775402 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103bed5-0334-43d8-ad9a-ece7fe29afb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:25.775888 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103bed5-0334-43d8-ad9a-ece7fe29afb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:25.778311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103bed5-0334-43d8-ad9a-ece7fe29afb4 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:26.001796 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c07819-a15a-4b77-94b4-7f4c97e77e06 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:26.001796 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0fc72-4acb-4ca7-adf7-f52734268abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:26.001796 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0fc72-4acb-4ca7-adf7-f52734268abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:26.001796 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0fc72-4acb-4ca7-adf7-f52734268abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:26.003630 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0fc72-4acb-4ca7-adf7-f52734268abf admin admin] PATH_INFO: `/v3/users/966aa03d88c143efbccb15cb25015478` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:26.003630 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff0fc72-4acb-4ca7-adf7-f52734268abf admin admin] RBAC: Authorizing `identity:delete_user(user_id=966aa03d88c143efbccb15cb25015478)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:26.003630 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff0fc72-4acb-4ca7-adf7-f52734268abf admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:26.217545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 907/1805] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:52:25 2022] POST /identity/v3/projects => generated 385 bytes in 805 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:52:26.276289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103bed5-0334-43d8-ad9a-ece7fe29afb4 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:26.279105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-0103bed5-0334-43d8-ad9a-ece7fe29afb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:26.367422 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103bed5-0334-43d8-ad9a-ece7fe29afb4 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:26.367422 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca8b66-935f-438e-b6c1-baed9fd7d011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:26.367422 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca8b66-935f-438e-b6c1-baed9fd7d011 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:26.367422 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca8b66-935f-438e-b6c1-baed9fd7d011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:26.367781 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca8b66-935f-438e-b6c1-baed9fd7d011 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:26.367781 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca8b66-935f-438e-b6c1-baed9fd7d011 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:26.368712 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca8b66-935f-438e-b6c1-baed9fd7d011 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:26.509791 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-5ff0fc72-4acb-4ca7-adf7-f52734268abf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '966aa03d88c143efbccb15cb25015478'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:26.533899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 908/1806] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:52:26 2022] POST /identity/v3/users => generated 293 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:52:26.536256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca8b66-935f-438e-b6c1-baed9fd7d011 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:26.562703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca8b66-935f-438e-b6c1-baed9fd7d011 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:26.563954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6d695-8c6a-43d6-8460-edf3d02053af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:26.564870 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6d695-8c6a-43d6-8460-edf3d02053af admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:26.565329 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6d695-8c6a-43d6-8460-edf3d02053af admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:26.565539 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6d695-8c6a-43d6-8460-edf3d02053af admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:26.566421 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6d695-8c6a-43d6-8460-edf3d02053af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:26.567979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6d695-8c6a-43d6-8460-edf3d02053af admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:26.584899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 909/1807] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:26.586667 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6d695-8c6a-43d6-8460-edf3d02053af admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:26.602453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6d695-8c6a-43d6-8460-edf3d02053af admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:26.607556 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a6748-14db-4fca-be11-3ca320015ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:26.608193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a6748-14db-4fca-be11-3ca320015ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:26.608502 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a6748-14db-4fca-be11-3ca320015ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:26.608771 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a6748-14db-4fca-be11-3ca320015ab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:26.609430 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a6748-14db-4fca-be11-3ca320015ab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:26.611390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a6748-14db-4fca-be11-3ca320015ab9 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:26.623509 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 910/1808] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:26.630310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-5ff0fc72-4acb-4ca7-adf7-f52734268abf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '966aa03d88c143efbccb15cb25015478'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:26.630674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a6748-14db-4fca-be11-3ca320015ab9 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:26.631014 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-5ff0fc72-4acb-4ca7-adf7-f52734268abf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '966aa03d88c143efbccb15cb25015478'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:26.633674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-8c6a6748-14db-4fca-be11-3ca320015ab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:26.642546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-5ff0fc72-4acb-4ca7-adf7-f52734268abf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '966aa03d88c143efbccb15cb25015478'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:26.643414 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-5ff0fc72-4acb-4ca7-adf7-f52734268abf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '966aa03d88c143efbccb15cb25015478'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:26.651834 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a6748-14db-4fca-be11-3ca320015ab9 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:26.669368 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6749ee-dded-4122-aa3e-a8dd90159d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:26.670894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6749ee-dded-4122-aa3e-a8dd90159d50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:26.671497 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6749ee-dded-4122-aa3e-a8dd90159d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:26.672040 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6749ee-dded-4122-aa3e-a8dd90159d50 admin admin] PATH_INFO: `/v3/projects/19fd914a0d5742148a0710a849c19355/users/0ca3a2fe7f8c4c82b0c7543cd0c18f12/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:26.676430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 899/1809] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:52:25 2022] DELETE /identity/v3/users/966aa03d88c143efbccb15cb25015478 => generated 0 bytes in 945 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:26.693770 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6749ee-dded-4122-aa3e-a8dd90159d50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19fd914a0d5742148a0710a849c19355, user_id=0ca3a2fe7f8c4c82b0c7543cd0c18f12, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:26.697282 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6749ee-dded-4122-aa3e-a8dd90159d50 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:26.792437 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:26.809901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 911/1810] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:52:26 2022] PUT /identity/v3/projects/19fd914a0d5742148a0710a849c19355/users/0ca3a2fe7f8c4c82b0c7543cd0c18f12/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:26.816622 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae5915-8324-4997-9d22-97f2a85c2e7c None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:26.817354 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae5915-8324-4997-9d22-97f2a85c2e7c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:26.817354 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae5915-8324-4997-9d22-97f2a85c2e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:26.850124 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-a2ae5915-8324-4997-9d22-97f2a85c2e7c None None] MFA Rules not processed for user `0ca3a2fe7f8c4c82b0c7543cd0c18f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:26.850124 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-a2ae5915-8324-4997-9d22-97f2a85c2e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:26.946780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 900/1811] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:26.950336 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6749ee-dded-4122-aa3e-a8dd90159d50 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:27.063818 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6749ee-dded-4122-aa3e-a8dd90159d50 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:27.075987 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:27.076855 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff051f8-bc0d-46b8-b9a3-5331cfca91a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:27.078126 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff051f8-bc0d-46b8-b9a3-5331cfca91a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:27.081910 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff051f8-bc0d-46b8-b9a3-5331cfca91a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:27.084101 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff051f8-bc0d-46b8-b9a3-5331cfca91a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:27.085100 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff051f8-bc0d-46b8-b9a3-5331cfca91a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:27.089199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff051f8-bc0d-46b8-b9a3-5331cfca91a2 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:27.231578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 912/1812] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:52:26 2022] POST /identity/v3/projects => generated 385 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:52:27.234537 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae5915-8324-4997-9d22-97f2a85c2e7c None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:27.236029 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-a2ae5915-8324-4997-9d22-97f2a85c2e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:27.237493 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff051f8-bc0d-46b8-b9a3-5331cfca91a2 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:27.257212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae5915-8324-4997-9d22-97f2a85c2e7c None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:27.341708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f82bd-bc5a-4444-8051-bc478ba3252c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:27.342680 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f82bd-bc5a-4444-8051-bc478ba3252c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:27.343050 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f82bd-bc5a-4444-8051-bc478ba3252c admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:27.343391 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f82bd-bc5a-4444-8051-bc478ba3252c admin admin] PATH_INFO: `/v3/projects/3fc1c830b81a42d0ac2e5100b36a23b8` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:27.344782 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f82bd-bc5a-4444-8051-bc478ba3252c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fc1c830b81a42d0ac2e5100b36a23b8)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:27.346857 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f82bd-bc5a-4444-8051-bc478ba3252c admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:27.357332 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff051f8-bc0d-46b8-b9a3-5331cfca91a2 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:27.366010 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25ee77-97a3-47af-acfc-fded1b90adc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:27.367764 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25ee77-97a3-47af-acfc-fded1b90adc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:27.368266 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25ee77-97a3-47af-acfc-fded1b90adc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:27.368751 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25ee77-97a3-47af-acfc-fded1b90adc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:27.370140 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25ee77-97a3-47af-acfc-fded1b90adc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:27.375053 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25ee77-97a3-47af-acfc-fded1b90adc9 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:27.581177 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 913/1813] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:52:27 2022] POST /identity/v3/users => generated 299 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:52:27.585389 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25ee77-97a3-47af-acfc-fded1b90adc9 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:27.597284 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25ee77-97a3-47af-acfc-fded1b90adc9 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:27.607707 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db944e09-bab0-4f72-91ac-06676c9ba495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:27.608804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db944e09-bab0-4f72-91ac-06676c9ba495 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:27.609298 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db944e09-bab0-4f72-91ac-06676c9ba495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:27.609800 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db944e09-bab0-4f72-91ac-06676c9ba495 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:27.610776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db944e09-bab0-4f72-91ac-06676c9ba495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:27.613095 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db944e09-bab0-4f72-91ac-06676c9ba495 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:27.637484 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 914/1814] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:27.641258 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db944e09-bab0-4f72-91ac-06676c9ba495 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:27.686407 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db944e09-bab0-4f72-91ac-06676c9ba495 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:27.691765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-d15f82bd-bc5a-4444-8051-bc478ba3252c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fc1c830b81a42d0ac2e5100b36a23b8', 'request_id': 'req-d15f82bd-bc5a-4444-8051-bc478ba3252c'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:27.723105 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-d15f82bd-bc5a-4444-8051-bc478ba3252c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fc1c830b81a42d0ac2e5100b36a23b8', 'request_id': 'req-d15f82bd-bc5a-4444-8051-bc478ba3252c'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:27.735358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-d15f82bd-bc5a-4444-8051-bc478ba3252c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fc1c830b81a42d0ac2e5100b36a23b8', 'request_id': 'req-d15f82bd-bc5a-4444-8051-bc478ba3252c'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:27.826317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7724de1-864b-41d2-82e2-88600f373699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:27.827381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7724de1-864b-41d2-82e2-88600f373699 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:27.827697 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7724de1-864b-41d2-82e2-88600f373699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:27.827942 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7724de1-864b-41d2-82e2-88600f373699 admin admin] PATH_INFO: `/v3/projects/1aa81ad0a8af46cc94219d61e4f84c95/users/97742b9471b841eeb9aae97413a5e746/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:27.856975 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-d15f82bd-bc5a-4444-8051-bc478ba3252c admin admin] The token cache is being invalidate because project 3fc1c830b81a42d0ac2e5100b36a23b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:52:27.857567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-d15f82bd-bc5a-4444-8051-bc478ba3252c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:27.858428 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-d15f82bd-bc5a-4444-8051-bc478ba3252c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:27.859102 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7724de1-864b-41d2-82e2-88600f373699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aa81ad0a8af46cc94219d61e4f84c95, user_id=97742b9471b841eeb9aae97413a5e746, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:27.862628 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 901/1815] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:52:27 2022] DELETE /identity/v3/projects/3fc1c830b81a42d0ac2e5100b36a23b8 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:27.862997 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7724de1-864b-41d2-82e2-88600f373699 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:27.881566 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da6493-fae5-4995-a673-2906ff6cc5f9 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:27.881566 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da6493-fae5-4995-a673-2906ff6cc5f9 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:27.881566 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da6493-fae5-4995-a673-2906ff6cc5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:27.913476 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-a0da6493-fae5-4995-a673-2906ff6cc5f9 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:27.916096 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-a0da6493-fae5-4995-a673-2906ff6cc5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:27.952795 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 902/1816] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:28.171014 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa336d78-b98d-4822-9d52-fbe2f651b09f None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:28.171014 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa336d78-b98d-4822-9d52-fbe2f651b09f None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:28.171014 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa336d78-b98d-4822-9d52-fbe2f651b09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:28.177023 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-fa336d78-b98d-4822-9d52-fbe2f651b09f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:28.178932 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-fa336d78-b98d-4822-9d52-fbe2f651b09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:28.198552 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 903/1817] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:28.199874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa336d78-b98d-4822-9d52-fbe2f651b09f None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:28.210395 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa336d78-b98d-4822-9d52-fbe2f651b09f None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:28.215985 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce35799-c07d-4e96-8e17-7be865a75ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:28.216660 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce35799-c07d-4e96-8e17-7be865a75ded admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:28.216881 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce35799-c07d-4e96-8e17-7be865a75ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:28.217144 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce35799-c07d-4e96-8e17-7be865a75ded admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:28.217791 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce35799-c07d-4e96-8e17-7be865a75ded admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:52:28.218088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce35799-c07d-4e96-8e17-7be865a75ded admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:28.219860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce35799-c07d-4e96-8e17-7be865a75ded admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:28.233483 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 904/1818] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:52:28.245853 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce35799-c07d-4e96-8e17-7be865a75ded admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:28.259560 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce35799-c07d-4e96-8e17-7be865a75ded admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:28.275695 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31811362-82be-466d-9802-0a23dab0d818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:28.277023 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31811362-82be-466d-9802-0a23dab0d818 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:28.277604 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31811362-82be-466d-9802-0a23dab0d818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:28.280949 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31811362-82be-466d-9802-0a23dab0d818 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:28.282058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31811362-82be-466d-9802-0a23dab0d818 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:28.284401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31811362-82be-466d-9802-0a23dab0d818 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:28.376579 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 915/1819] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:52:27 2022] PUT /identity/v3/projects/1aa81ad0a8af46cc94219d61e4f84c95/users/97742b9471b841eeb9aae97413a5e746/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:28.383636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7724de1-864b-41d2-82e2-88600f373699 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:28.386353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-e7724de1-864b-41d2-82e2-88600f373699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:28.476694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 905/1820] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:52:28 2022] POST /identity/v3/projects => generated 375 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:52:28.487636 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31811362-82be-466d-9802-0a23dab0d818 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:28.632808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7724de1-864b-41d2-82e2-88600f373699 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:28.671698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31811362-82be-466d-9802-0a23dab0d818 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:28.706468 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719fa5d5-cf10-4702-847e-3a196c0df14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:28.707908 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fa5d5-cf10-4702-847e-3a196c0df14d admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:28.708547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fa5d5-cf10-4702-847e-3a196c0df14d admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:28.709171 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fa5d5-cf10-4702-847e-3a196c0df14d admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:28.710536 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719fa5d5-cf10-4702-847e-3a196c0df14d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:28.722800 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719fa5d5-cf10-4702-847e-3a196c0df14d admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:28.820820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285bf6d-0280-4b98-989a-5c04fdc6d3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:28.822893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285bf6d-0280-4b98-989a-5c04fdc6d3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:28.822893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285bf6d-0280-4b98-989a-5c04fdc6d3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:28.822893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285bf6d-0280-4b98-989a-5c04fdc6d3f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:28.823846 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5285bf6d-0280-4b98-989a-5c04fdc6d3f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:28.825620 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5285bf6d-0280-4b98-989a-5c04fdc6d3f2 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:28.843309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 916/1821] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 462 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:28.844481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285bf6d-0280-4b98-989a-5c04fdc6d3f2 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:28.863822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 906/1822] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:52:28 2022] POST /identity/v3/users => generated 288 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:52:28.865213 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285bf6d-0280-4b98-989a-5c04fdc6d3f2 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:28.870280 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719fa5d5-cf10-4702-847e-3a196c0df14d admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:28.892073 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43942654-4728-4f49-b294-2fadb7b06a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:28.893865 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43942654-4728-4f49-b294-2fadb7b06a8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:28.894175 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43942654-4728-4f49-b294-2fadb7b06a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:28.894838 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43942654-4728-4f49-b294-2fadb7b06a8a admin admin] PATH_INFO: `/v3/projects/1aa81ad0a8af46cc94219d61e4f84c95/users/97742b9471b841eeb9aae97413a5e746/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:28.908193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43942654-4728-4f49-b294-2fadb7b06a8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aa81ad0a8af46cc94219d61e4f84c95, user_id=97742b9471b841eeb9aae97413a5e746, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:28.910908 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719fa5d5-cf10-4702-847e-3a196c0df14d admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:28.914358 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43942654-4728-4f49-b294-2fadb7b06a8a admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:28.920874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803cd4e-1e3c-4dff-95bd-f65ce2fe76f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:28.936555 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803cd4e-1e3c-4dff-95bd-f65ce2fe76f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:28.937615 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803cd4e-1e3c-4dff-95bd-f65ce2fe76f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:28.938343 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803cd4e-1e3c-4dff-95bd-f65ce2fe76f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:28.939148 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 917/1823] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:52:28 2022] PUT /identity/v3/projects/1aa81ad0a8af46cc94219d61e4f84c95/users/97742b9471b841eeb9aae97413a5e746/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:28.939920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803cd4e-1e3c-4dff-95bd-f65ce2fe76f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:28.942180 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803cd4e-1e3c-4dff-95bd-f65ce2fe76f3 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:28.952582 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959a9d6-59c5-494c-9be5-0a872e2646b8 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:28.955613 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959a9d6-59c5-494c-9be5-0a872e2646b8 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:28.955613 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959a9d6-59c5-494c-9be5-0a872e2646b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:28.966013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 907/1824] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:28.987895 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803cd4e-1e3c-4dff-95bd-f65ce2fe76f3 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:29.002151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803cd4e-1e3c-4dff-95bd-f65ce2fe76f3 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:29.016229 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-a959a9d6-59c5-494c-9be5-0a872e2646b8 None None] MFA Rules not processed for user `97742b9471b841eeb9aae97413a5e746`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:29.018709 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-a959a9d6-59c5-494c-9be5-0a872e2646b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:29.165697 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4e479-1961-421b-b6b1-494267eec002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:29.166312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4e479-1961-421b-b6b1-494267eec002 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:29.166591 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4e479-1961-421b-b6b1-494267eec002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:29.166859 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4e479-1961-421b-b6b1-494267eec002 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:29.167548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4e479-1961-421b-b6b1-494267eec002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:29.172332 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4e479-1961-421b-b6b1-494267eec002 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:29.229070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 918/1825] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:29.230860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 908/1826] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:29.233007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959a9d6-59c5-494c-9be5-0a872e2646b8 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:29.250301 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4e479-1961-421b-b6b1-494267eec002 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:29.253177 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d6c4e479-1961-421b-b6b1-494267eec002 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:29.282018 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959a9d6-59c5-494c-9be5-0a872e2646b8 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:29.293723 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cbcf0a-183e-4778-bf01-13ea0e6bd3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:29.299216 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbcf0a-183e-4778-bf01-13ea0e6bd3e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:29.300264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbcf0a-183e-4778-bf01-13ea0e6bd3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:29.300264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbcf0a-183e-4778-bf01-13ea0e6bd3e3 admin admin] PATH_INFO: `/v3/projects/ff651eab623d4748b16914e5ff4941a1/users/5a1203cf3b194344822af1836c0a1d68/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:29.326383 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cbcf0a-183e-4778-bf01-13ea0e6bd3e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff651eab623d4748b16914e5ff4941a1, user_id=5a1203cf3b194344822af1836c0a1d68, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:29.332080 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cbcf0a-183e-4778-bf01-13ea0e6bd3e3 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:29.393310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4e479-1961-421b-b6b1-494267eec002 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:29.399756 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3b4fa-73d8-412f-9e32-17a71d06d91e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:29.400527 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3b4fa-73d8-412f-9e32-17a71d06d91e service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:29.400870 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3b4fa-73d8-412f-9e32-17a71d06d91e service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:29.401141 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3b4fa-73d8-412f-9e32-17a71d06d91e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:29.403215 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-e4f3b4fa-73d8-412f-9e32-17a71d06d91e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:29.473198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 919/1827] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:52:29 2022] PUT /identity/v3/projects/ff651eab623d4748b16914e5ff4941a1/users/5a1203cf3b194344822af1836c0a1d68/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:29.475793 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dc5f3-e586-4c8c-be5b-80d76ab13c14 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:29.476913 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dc5f3-e586-4c8c-be5b-80d76ab13c14 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:29.477654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dc5f3-e586-4c8c-be5b-80d76ab13c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:29.511831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3b4fa-73d8-412f-9e32-17a71d06d91e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:29.516256 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3b4fa-73d8-412f-9e32-17a71d06d91e service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:29.542940 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-f96dc5f3-e586-4c8c-be5b-80d76ab13c14 None None] MFA Rules not processed for user `5a1203cf3b194344822af1836c0a1d68`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:29.556328 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f96dc5f3-e586-4c8c-be5b-80d76ab13c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:29.556744 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 909/1828] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:52:29.657726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 920/1829] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:29.662722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3b4fa-73d8-412f-9e32-17a71d06d91e service cinder] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:29.684571 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3b4fa-73d8-412f-9e32-17a71d06d91e service cinder] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:29.774592 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b177f-5a2c-4b6b-9b54-6205ae740c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:29.775549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b177f-5a2c-4b6b-9b54-6205ae740c59 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:29.775899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b177f-5a2c-4b6b-9b54-6205ae740c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:29.776271 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b177f-5a2c-4b6b-9b54-6205ae740c59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:29.777104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b177f-5a2c-4b6b-9b54-6205ae740c59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:29.779417 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b177f-5a2c-4b6b-9b54-6205ae740c59 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:29.861247 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 910/1830] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:52:29 2022] POST /identity/v3/projects => generated 375 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:52:29.865919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dc5f3-e586-4c8c-be5b-80d76ab13c14 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:29.884966 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dc5f3-e586-4c8c-be5b-80d76ab13c14 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:29.980233 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d749e-720b-45b2-98ea-4b09d426a936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:29.980892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d749e-720b-45b2-98ea-4b09d426a936 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:29.981233 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d749e-720b-45b2-98ea-4b09d426a936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:29.981521 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d749e-720b-45b2-98ea-4b09d426a936 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:29.982206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d749e-720b-45b2-98ea-4b09d426a936 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:29.984300 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d749e-720b-45b2-98ea-4b09d426a936 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:30.139314 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 921/1831] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:52:29 2022] POST /identity/v3/users => generated 294 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:52:30.145075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b177f-5a2c-4b6b-9b54-6205ae740c59 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:30.157931 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b177f-5a2c-4b6b-9b54-6205ae740c59 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:30.163634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78fb4e-a1e7-42ea-ad76-95b5c0141009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:30.164583 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78fb4e-a1e7-42ea-ad76-95b5c0141009 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:30.164972 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78fb4e-a1e7-42ea-ad76-95b5c0141009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:30.165288 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78fb4e-a1e7-42ea-ad76-95b5c0141009 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:30.166074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce78fb4e-a1e7-42ea-ad76-95b5c0141009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:30.168273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce78fb4e-a1e7-42ea-ad76-95b5c0141009 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:30.180932 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 911/1832] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:30.202539 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d749e-720b-45b2-98ea-4b09d426a936 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:30.214075 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d749e-720b-45b2-98ea-4b09d426a936 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:30.219331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f148da-348d-4367-9751-c6b3ff044096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:30.220030 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f148da-348d-4367-9751-c6b3ff044096 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:30.220415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f148da-348d-4367-9751-c6b3ff044096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:30.220719 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f148da-348d-4367-9751-c6b3ff044096 admin admin] PATH_INFO: `/v3/projects/b73dd6be895544749e1bc2e901770191/users/612ed8f6a9694d739a975e8e7d8356da/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:30.239823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f148da-348d-4367-9751-c6b3ff044096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b73dd6be895544749e1bc2e901770191, user_id=612ed8f6a9694d739a975e8e7d8356da, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:30.242707 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f148da-348d-4367-9751-c6b3ff044096 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:30.504815 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 922/1833] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:52:30 2022] PUT /identity/v3/projects/b73dd6be895544749e1bc2e901770191/users/612ed8f6a9694d739a975e8e7d8356da/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:52:30.507277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78fb4e-a1e7-42ea-ad76-95b5c0141009 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:30.529073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78fb4e-a1e7-42ea-ad76-95b5c0141009 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:30.634859 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b9f78-c154-4c1a-9897-0936a9ea5a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:30.635496 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b9f78-c154-4c1a-9897-0936a9ea5a46 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:30.635791 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b9f78-c154-4c1a-9897-0936a9ea5a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:30.636054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b9f78-c154-4c1a-9897-0936a9ea5a46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:30.637166 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b9f78-c154-4c1a-9897-0936a9ea5a46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:30.638375 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b9f78-c154-4c1a-9897-0936a9ea5a46 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:30.651079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 912/1834] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:30.658924 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:30.658924 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f148da-348d-4367-9751-c6b3ff044096 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:30.673604 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f148da-348d-4367-9751-c6b3ff044096 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:30.685398 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ceed06-274d-4db8-8273-bef222444ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:30.687700 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ceed06-274d-4db8-8273-bef222444ed4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:30.687700 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ceed06-274d-4db8-8273-bef222444ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:30.688317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ceed06-274d-4db8-8273-bef222444ed4 admin admin] PATH_INFO: `/v3/projects/b73dd6be895544749e1bc2e901770191/users/612ed8f6a9694d739a975e8e7d8356da/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:30.692917 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ceed06-274d-4db8-8273-bef222444ed4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b73dd6be895544749e1bc2e901770191, user_id=612ed8f6a9694d739a975e8e7d8356da, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:30.702419 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ceed06-274d-4db8-8273-bef222444ed4 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:30.713316 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 923/1835] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:52:30 2022] PUT /identity/v3/projects/b73dd6be895544749e1bc2e901770191/users/612ed8f6a9694d739a975e8e7d8356da/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:52:30.724378 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbc0ce-42c5-4da4-b8cd-8c738b28539a None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:30.725386 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbc0ce-42c5-4da4-b8cd-8c738b28539a None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:30.730959 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbc0ce-42c5-4da4-b8cd-8c738b28539a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:30.767862 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-3dfbc0ce-42c5-4da4-b8cd-8c738b28539a None None] MFA Rules not processed for user `612ed8f6a9694d739a975e8e7d8356da`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:30.769124 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-3dfbc0ce-42c5-4da4-b8cd-8c738b28539a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:30.869637 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 913/1836] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:30.884231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ceed06-274d-4db8-8273-bef222444ed4 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:30.950613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:30.970764 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ceed06-274d-4db8-8273-bef222444ed4 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:30.975186 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd154a71-0201-4ebd-a023-30b75c7ca298 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:30.975834 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd154a71-0201-4ebd-a023-30b75c7ca298 service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:30.976137 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd154a71-0201-4ebd-a023-30b75c7ca298 service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:30.976414 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd154a71-0201-4ebd-a023-30b75c7ca298 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:31.052034 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd154a71-0201-4ebd-a023-30b75c7ca298 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:31.053525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd154a71-0201-4ebd-a023-30b75c7ca298 service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:31.064633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 924/1837] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:31.122751 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:33.240634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbc0ce-42c5-4da4-b8cd-8c738b28539a None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:33.260588 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbc0ce-42c5-4da4-b8cd-8c738b28539a None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:33.265877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08412c-1694-42e3-b1e0-54006a936f1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:33.266900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08412c-1694-42e3-b1e0-54006a936f1e service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:33.267318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08412c-1694-42e3-b1e0-54006a936f1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:33.267740 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08412c-1694-42e3-b1e0-54006a936f1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:33.270277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-5a08412c-1694-42e3-b1e0-54006a936f1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:33.324129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:33.378411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08412c-1694-42e3-b1e0-54006a936f1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:33.380218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08412c-1694-42e3-b1e0-54006a936f1e service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:33.394294 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 914/1838] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:33.713635 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:35.236471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:36.340467 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd154a71-0201-4ebd-a023-30b75c7ca298 service cinder] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:36.340467 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-fd154a71-0201-4ebd-a023-30b75c7ca298 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:36.353784 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd154a71-0201-4ebd-a023-30b75c7ca298 service cinder] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:36.372921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f7319-2458-4123-b587-68dc176a80c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:36.378872 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f7319-2458-4123-b587-68dc176a80c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:36.379287 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f7319-2458-4123-b587-68dc176a80c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:36.392454 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f7319-2458-4123-b587-68dc176a80c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:36.528516 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f7319-2458-4123-b587-68dc176a80c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:36.530047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f7319-2458-4123-b587-68dc176a80c9 service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:36.548744 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 925/1839] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:36 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:36.789240 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:36.962148 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:37.659479 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:37.885520 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:39.099171 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:39.145203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08412c-1694-42e3-b1e0-54006a936f1e service cinder] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:39.156901 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08412c-1694-42e3-b1e0-54006a936f1e service cinder] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:39.161756 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69116fd5-49cb-4d9d-80d4-bcb6ca230436 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:39.162641 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69116fd5-49cb-4d9d-80d4-bcb6ca230436 service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:39.163110 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69116fd5-49cb-4d9d-80d4-bcb6ca230436 service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:39.164187 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69116fd5-49cb-4d9d-80d4-bcb6ca230436 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:39.191451 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69116fd5-49cb-4d9d-80d4-bcb6ca230436 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:39.191451 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69116fd5-49cb-4d9d-80d4-bcb6ca230436 service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:39.200823 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 915/1840] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:52:42.188261 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98917d1-901a-4201-9ea7-24f76d260b99 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:42.188804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98917d1-901a-4201-9ea7-24f76d260b99 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:42.189565 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98917d1-901a-4201-9ea7-24f76d260b99 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:42.193566 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 926/1841] 10.222.0.13 () {54 vars in 921 bytes} [Mon Aug 8 16:52:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:52:42.235923 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f1ac7-d593-4163-bac3-f8a978fba3b5 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:42.236743 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f1ac7-d593-4163-bac3-f8a978fba3b5 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:42.237022 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f1ac7-d593-4163-bac3-f8a978fba3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:42.279016 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-969f1ac7-d593-4163-bac3-f8a978fba3b5 None None] MFA Rules not processed for user `9daa7489bced467984d39e8b8ac49d13`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:42.281791 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-969f1ac7-d593-4163-bac3-f8a978fba3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:42.474995 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 916/1842] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Aug 8 16:52:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:52:42.499396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98917d1-901a-4201-9ea7-24f76d260b99 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:42.502469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b98917d1-901a-4201-9ea7-24f76d260b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:42.529997 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98917d1-901a-4201-9ea7-24f76d260b99 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:42.542273 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bb0c7-b60c-42c1-a297-2c05c9de5c20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:42.543750 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bb0c7-b60c-42c1-a297-2c05c9de5c20 service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:42.544777 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bb0c7-b60c-42c1-a297-2c05c9de5c20 service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:42.545604 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bb0c7-b60c-42c1-a297-2c05c9de5c20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:42.565415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bb0c7-b60c-42c1-a297-2c05c9de5c20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:42.568440 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bb0c7-b60c-42c1-a297-2c05c9de5c20 service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:42.610894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 927/1843] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:46.291008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f1ac7-d593-4163-bac3-f8a978fba3b5 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:46.378789 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f1ac7-d593-4163-bac3-f8a978fba3b5 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:46.385638 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12aeaa-ba77-4635-85dd-9e6f0db6f721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:46.386468 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12aeaa-ba77-4635-85dd-9e6f0db6f721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:46.386900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12aeaa-ba77-4635-85dd-9e6f0db6f721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:46.387258 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12aeaa-ba77-4635-85dd-9e6f0db6f721 admin admin] PATH_INFO: `/v3/users/5f0008eec3394de093222264494ea4f0` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:46.388674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12aeaa-ba77-4635-85dd-9e6f0db6f721 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f0008eec3394de093222264494ea4f0)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:46.391178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12aeaa-ba77-4635-85dd-9e6f0db6f721 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:46.408641 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:46.661240 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-cc12aeaa-ba77-4635-85dd-9e6f0db6f721 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f0008eec3394de093222264494ea4f0'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:46.788717 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-cc12aeaa-ba77-4635-85dd-9e6f0db6f721 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f0008eec3394de093222264494ea4f0'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:46.789571 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-cc12aeaa-ba77-4635-85dd-9e6f0db6f721 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f0008eec3394de093222264494ea4f0'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:46.802613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-cc12aeaa-ba77-4635-85dd-9e6f0db6f721 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f0008eec3394de093222264494ea4f0'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:46.803547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-cc12aeaa-ba77-4635-85dd-9e6f0db6f721 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f0008eec3394de093222264494ea4f0'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:46.841607 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 917/1844] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:52:46 2022] DELETE /identity/v3/users/5f0008eec3394de093222264494ea4f0 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:46.852293 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51941d4-d6bf-46c6-bed6-f2479b4ebb08 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:46.852293 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51941d4-d6bf-46c6-bed6-f2479b4ebb08 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:46.852293 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51941d4-d6bf-46c6-bed6-f2479b4ebb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:46.874301 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-a51941d4-d6bf-46c6-bed6-f2479b4ebb08 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:46.876578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-a51941d4-d6bf-46c6-bed6-f2479b4ebb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:46.885655 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:47.014323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 928/1845] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:47.024487 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12aeaa-ba77-4635-85dd-9e6f0db6f721 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:47.026913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-cc12aeaa-ba77-4635-85dd-9e6f0db6f721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:47.225295 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12aeaa-ba77-4635-85dd-9e6f0db6f721 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:47.246725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b99e3-c154-4876-904b-1f57b82708a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:47.247456 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b99e3-c154-4876-904b-1f57b82708a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:47.247741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b99e3-c154-4876-904b-1f57b82708a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:47.249736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b99e3-c154-4876-904b-1f57b82708a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:47.263434 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5b99e3-c154-4876-904b-1f57b82708a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:47.266029 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5b99e3-c154-4876-904b-1f57b82708a7 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:47.293124 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 918/1846] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:48.484592 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51941d4-d6bf-46c6-bed6-f2479b4ebb08 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:48.486353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-a51941d4-d6bf-46c6-bed6-f2479b4ebb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:48.506809 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51941d4-d6bf-46c6-bed6-f2479b4ebb08 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:48.515655 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec390c-70b2-4473-9877-cd782bbe3ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:48.517802 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec390c-70b2-4473-9877-cd782bbe3ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:48.518381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec390c-70b2-4473-9877-cd782bbe3ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:48.518900 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec390c-70b2-4473-9877-cd782bbe3ac0 admin admin] PATH_INFO: `/v3/projects/5e4d79d7beca4f9292e13c51d9633317` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:48.521170 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec390c-70b2-4473-9877-cd782bbe3ac0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e4d79d7beca4f9292e13c51d9633317)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:48.525680 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec390c-70b2-4473-9877-cd782bbe3ac0 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:48.705210 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-f9ec390c-70b2-4473-9877-cd782bbe3ac0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e4d79d7beca4f9292e13c51d9633317', 'request_id': 'req-f9ec390c-70b2-4473-9877-cd782bbe3ac0'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:48.774382 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-f9ec390c-70b2-4473-9877-cd782bbe3ac0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e4d79d7beca4f9292e13c51d9633317', 'request_id': 'req-f9ec390c-70b2-4473-9877-cd782bbe3ac0'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:48.785869 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-f9ec390c-70b2-4473-9877-cd782bbe3ac0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e4d79d7beca4f9292e13c51d9633317', 'request_id': 'req-f9ec390c-70b2-4473-9877-cd782bbe3ac0'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:48.813814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-f9ec390c-70b2-4473-9877-cd782bbe3ac0 admin admin] The token cache is being invalidate because project 5e4d79d7beca4f9292e13c51d9633317 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:52:48.813814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-f9ec390c-70b2-4473-9877-cd782bbe3ac0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:48.813814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-f9ec390c-70b2-4473-9877-cd782bbe3ac0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:48.813814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 929/1847] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:52:48 2022] DELETE /identity/v3/projects/5e4d79d7beca4f9292e13c51d9633317 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:48.817968 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b99e3-c154-4876-904b-1f57b82708a7 service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:48.834358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b99e3-c154-4876-904b-1f57b82708a7 service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:48.932416 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddd3ce-0549-4199-80d9-9d45f15dc741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:48.933208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddd3ce-0549-4199-80d9-9d45f15dc741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:48.933779 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddd3ce-0549-4199-80d9-9d45f15dc741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:48.934154 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddd3ce-0549-4199-80d9-9d45f15dc741 admin admin] PATH_INFO: `/v3/users/0ca3a2fe7f8c4c82b0c7543cd0c18f12` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:48.936349 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ddd3ce-0549-4199-80d9-9d45f15dc741 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ca3a2fe7f8c4c82b0c7543cd0c18f12)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:48.939241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ddd3ce-0549-4199-80d9-9d45f15dc741 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:49.172949 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:49.189818 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-33ddd3ce-0549-4199-80d9-9d45f15dc741 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ca3a2fe7f8c4c82b0c7543cd0c18f12'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:49.293642 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-33ddd3ce-0549-4199-80d9-9d45f15dc741 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ca3a2fe7f8c4c82b0c7543cd0c18f12'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:49.294266 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-33ddd3ce-0549-4199-80d9-9d45f15dc741 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ca3a2fe7f8c4c82b0c7543cd0c18f12'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:49.307921 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-33ddd3ce-0549-4199-80d9-9d45f15dc741 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ca3a2fe7f8c4c82b0c7543cd0c18f12'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:49.309164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-33ddd3ce-0549-4199-80d9-9d45f15dc741 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ca3a2fe7f8c4c82b0c7543cd0c18f12'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:49.343492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 919/1848] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:52:48 2022] DELETE /identity/v3/users/0ca3a2fe7f8c4c82b0c7543cd0c18f12 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:52:50.013352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec390c-70b2-4473-9877-cd782bbe3ac0 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:50.016882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f9ec390c-70b2-4473-9877-cd782bbe3ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:50.044443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec390c-70b2-4473-9877-cd782bbe3ac0 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:50.159625 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ef4c4-56e3-43d9-a64e-476787cf2f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:50.160596 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ef4c4-56e3-43d9-a64e-476787cf2f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:50.160992 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ef4c4-56e3-43d9-a64e-476787cf2f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:50.161339 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ef4c4-56e3-43d9-a64e-476787cf2f16 admin admin] PATH_INFO: `/v3/projects/19fd914a0d5742148a0710a849c19355` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:50.162888 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ef4c4-56e3-43d9-a64e-476787cf2f16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19fd914a0d5742148a0710a849c19355)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:50.166156 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ef4c4-56e3-43d9-a64e-476787cf2f16 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:50.302958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:50.314532 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-f12ef4c4-56e3-43d9-a64e-476787cf2f16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19fd914a0d5742148a0710a849c19355', 'request_id': 'req-f12ef4c4-56e3-43d9-a64e-476787cf2f16'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:50.396051 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-f12ef4c4-56e3-43d9-a64e-476787cf2f16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19fd914a0d5742148a0710a849c19355', 'request_id': 'req-f12ef4c4-56e3-43d9-a64e-476787cf2f16'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:50.405037 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-f12ef4c4-56e3-43d9-a64e-476787cf2f16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19fd914a0d5742148a0710a849c19355', 'request_id': 'req-f12ef4c4-56e3-43d9-a64e-476787cf2f16'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:50.418368 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-f12ef4c4-56e3-43d9-a64e-476787cf2f16 admin admin] The token cache is being invalidate because project 19fd914a0d5742148a0710a849c19355 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:52:50.418767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-f12ef4c4-56e3-43d9-a64e-476787cf2f16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:50.419380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-f12ef4c4-56e3-43d9-a64e-476787cf2f16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:50.421020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 930/1849] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:52:50 2022] DELETE /identity/v3/projects/19fd914a0d5742148a0710a849c19355 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:50.425867 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddd3ce-0549-4199-80d9-9d45f15dc741 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:50.427504 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-33ddd3ce-0549-4199-80d9-9d45f15dc741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:50.441724 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddd3ce-0549-4199-80d9-9d45f15dc741 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:50.536160 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed21b8-be9e-4b35-a919-68c07c886143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:50.537101 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed21b8-be9e-4b35-a919-68c07c886143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:50.537595 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed21b8-be9e-4b35-a919-68c07c886143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:50.537978 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed21b8-be9e-4b35-a919-68c07c886143 admin admin] PATH_INFO: `/v3/users/97742b9471b841eeb9aae97413a5e746` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:50.540315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed21b8-be9e-4b35-a919-68c07c886143 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97742b9471b841eeb9aae97413a5e746)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:50.543198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed21b8-be9e-4b35-a919-68c07c886143 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:50.770213 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-50ed21b8-be9e-4b35-a919-68c07c886143 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97742b9471b841eeb9aae97413a5e746'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:50.836492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-50ed21b8-be9e-4b35-a919-68c07c886143 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97742b9471b841eeb9aae97413a5e746'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:50.837739 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-50ed21b8-be9e-4b35-a919-68c07c886143 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97742b9471b841eeb9aae97413a5e746'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:50.855777 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-50ed21b8-be9e-4b35-a919-68c07c886143 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97742b9471b841eeb9aae97413a5e746'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:50.857665 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-50ed21b8-be9e-4b35-a919-68c07c886143 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97742b9471b841eeb9aae97413a5e746'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:50.908937 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 920/1850] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:52:50 2022] DELETE /identity/v3/users/97742b9471b841eeb9aae97413a5e746 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:51.630502 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ef4c4-56e3-43d9-a64e-476787cf2f16 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:51.633614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f12ef4c4-56e3-43d9-a64e-476787cf2f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:51.658094 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ef4c4-56e3-43d9-a64e-476787cf2f16 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:51.756170 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bbbb26-a3f3-463e-a0aa-4fa069894e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:51.757559 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bbbb26-a3f3-463e-a0aa-4fa069894e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:51.757953 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bbbb26-a3f3-463e-a0aa-4fa069894e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:51.758361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bbbb26-a3f3-463e-a0aa-4fa069894e66 admin admin] PATH_INFO: `/v3/projects/1aa81ad0a8af46cc94219d61e4f84c95` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:51.759893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bbbb26-a3f3-463e-a0aa-4fa069894e66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aa81ad0a8af46cc94219d61e4f84c95)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:51.762657 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bbbb26-a3f3-463e-a0aa-4fa069894e66 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:51.931386 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-62bbbb26-a3f3-463e-a0aa-4fa069894e66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aa81ad0a8af46cc94219d61e4f84c95', 'request_id': 'req-62bbbb26-a3f3-463e-a0aa-4fa069894e66'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:52.005379 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-62bbbb26-a3f3-463e-a0aa-4fa069894e66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aa81ad0a8af46cc94219d61e4f84c95', 'request_id': 'req-62bbbb26-a3f3-463e-a0aa-4fa069894e66'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:52.013902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-62bbbb26-a3f3-463e-a0aa-4fa069894e66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aa81ad0a8af46cc94219d61e4f84c95', 'request_id': 'req-62bbbb26-a3f3-463e-a0aa-4fa069894e66'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:52.026514 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-62bbbb26-a3f3-463e-a0aa-4fa069894e66 admin admin] The token cache is being invalidate because project 1aa81ad0a8af46cc94219d61e4f84c95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:52:52.026837 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-62bbbb26-a3f3-463e-a0aa-4fa069894e66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:52.027798 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-62bbbb26-a3f3-463e-a0aa-4fa069894e66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:52.031125 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 931/1851] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:52:51 2022] DELETE /identity/v3/projects/1aa81ad0a8af46cc94219d61e4f84c95 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:52.037467 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff8b18-f358-426b-8ae0-e2c5c5e52d1c None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:52.040928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff8b18-f358-426b-8ae0-e2c5c5e52d1c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:52.076349 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff8b18-f358-426b-8ae0-e2c5c5e52d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:52.081263 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-bcff8b18-f358-426b-8ae0-e2c5c5e52d1c None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:52.083098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-bcff8b18-f358-426b-8ae0-e2c5c5e52d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:52.161371 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:52.213288 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 921/1852] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:52.226063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f496b-ecd3-4cf8-b7b1-9e68805ab6a5 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:52.226554 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f496b-ecd3-4cf8-b7b1-9e68805ab6a5 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:52.227299 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f496b-ecd3-4cf8-b7b1-9e68805ab6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:52.275918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-6b7f496b-ecd3-4cf8-b7b1-9e68805ab6a5 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:52.277342 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-6b7f496b-ecd3-4cf8-b7b1-9e68805ab6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:52.298712 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 932/1853] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:52.301647 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff8b18-f358-426b-8ae0-e2c5c5e52d1c None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:52.327664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff8b18-f358-426b-8ae0-e2c5c5e52d1c None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:52.338471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f83d00-d4c0-45ff-aa11-54e7aa07ae35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:52.340289 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f83d00-d4c0-45ff-aa11-54e7aa07ae35 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:52.349468 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f83d00-d4c0-45ff-aa11-54e7aa07ae35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:52.349931 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f83d00-d4c0-45ff-aa11-54e7aa07ae35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:52.350949 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f83d00-d4c0-45ff-aa11-54e7aa07ae35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:52:52.351410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f83d00-d4c0-45ff-aa11-54e7aa07ae35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:52.353243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f83d00-d4c0-45ff-aa11-54e7aa07ae35 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:52.384698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 922/1854] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:52:52.388642 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f496b-ecd3-4cf8-b7b1-9e68805ab6a5 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:52.409820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f496b-ecd3-4cf8-b7b1-9e68805ab6a5 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:52.418317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e7995-4a39-4c87-b72a-08e311ea64f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:52.419317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e7995-4a39-4c87-b72a-08e311ea64f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:52.419809 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e7995-4a39-4c87-b72a-08e311ea64f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:52.420279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e7995-4a39-4c87-b72a-08e311ea64f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:52.421355 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e7995-4a39-4c87-b72a-08e311ea64f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:52.424508 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e7995-4a39-4c87-b72a-08e311ea64f4 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:52.541346 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 933/1855] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:52:52 2022] POST /identity/v3/projects => generated 371 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:52:52.546029 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f83d00-d4c0-45ff-aa11-54e7aa07ae35 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:52.558205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f83d00-d4c0-45ff-aa11-54e7aa07ae35 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:52.653640 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3ecda-f1c2-4516-b08f-3d5a9ed58d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:52.654970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3ecda-f1c2-4516-b08f-3d5a9ed58d85 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:52.655505 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3ecda-f1c2-4516-b08f-3d5a9ed58d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:52.655943 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3ecda-f1c2-4516-b08f-3d5a9ed58d85 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:52.657241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3ecda-f1c2-4516-b08f-3d5a9ed58d85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:52.659823 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3ecda-f1c2-4516-b08f-3d5a9ed58d85 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:52.797719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 923/1856] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:52:52 2022] POST /identity/v3/users => generated 286 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:52:52.803014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e7995-4a39-4c87-b72a-08e311ea64f4 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:52.821961 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e7995-4a39-4c87-b72a-08e311ea64f4 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:52.829531 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b261c67-0156-4feb-9e0b-003a9c30b2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:52.839401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b261c67-0156-4feb-9e0b-003a9c30b2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:52.839401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b261c67-0156-4feb-9e0b-003a9c30b2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:52.839401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b261c67-0156-4feb-9e0b-003a9c30b2e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:52.839401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b261c67-0156-4feb-9e0b-003a9c30b2e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:52.839401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b261c67-0156-4feb-9e0b-003a9c30b2e4 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:52.849525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 934/1857] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:52.855288 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3ecda-f1c2-4516-b08f-3d5a9ed58d85 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:52.875524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3ecda-f1c2-4516-b08f-3d5a9ed58d85 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:52.884138 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b562895-e087-4f23-bc5a-f38afd3b5283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:52.885173 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b562895-e087-4f23-bc5a-f38afd3b5283 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:52.885659 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b562895-e087-4f23-bc5a-f38afd3b5283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:52.886107 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b562895-e087-4f23-bc5a-f38afd3b5283 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:52.887387 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b562895-e087-4f23-bc5a-f38afd3b5283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:52.890473 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b562895-e087-4f23-bc5a-f38afd3b5283 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:52.907072 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 924/1858] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:52:52.912579 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b261c67-0156-4feb-9e0b-003a9c30b2e4 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:52.925818 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b261c67-0156-4feb-9e0b-003a9c30b2e4 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:52.939295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d548fd9b-1b3b-42b6-8d15-ed4510e80133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:52.940453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548fd9b-1b3b-42b6-8d15-ed4510e80133 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:52.940800 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548fd9b-1b3b-42b6-8d15-ed4510e80133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:52.941172 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548fd9b-1b3b-42b6-8d15-ed4510e80133 admin admin] PATH_INFO: `/v3/projects/ec57970389084368ac985ecb58645bd1/users/00fdea0388e34b1eb5bfff49a3de5b57/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:52.972384 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:52.981583 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d548fd9b-1b3b-42b6-8d15-ed4510e80133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec57970389084368ac985ecb58645bd1, user_id=00fdea0388e34b1eb5bfff49a3de5b57, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:52.987336 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d548fd9b-1b3b-42b6-8d15-ed4510e80133 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:53.077851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 935/1859] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:52:52 2022] PUT /identity/v3/projects/ec57970389084368ac985ecb58645bd1/users/00fdea0388e34b1eb5bfff49a3de5b57/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:52:53.084186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7c4ec-4155-4c1e-958a-febc0adb7613 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:53.085048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7c4ec-4155-4c1e-958a-febc0adb7613 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:53.085048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7c4ec-4155-4c1e-958a-febc0adb7613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:53.108535 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-02c7c4ec-4155-4c1e-958a-febc0adb7613 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:53.109818 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-02c7c4ec-4155-4c1e-958a-febc0adb7613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:53.219139 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 925/1860] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:53.240525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d548fd9b-1b3b-42b6-8d15-ed4510e80133 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:53.245327 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-d548fd9b-1b3b-42b6-8d15-ed4510e80133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:53.358522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d548fd9b-1b3b-42b6-8d15-ed4510e80133 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:53.364144 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58c5f2-6381-402c-933a-4c89309be637 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:53.365220 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58c5f2-6381-402c-933a-4c89309be637 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:53.365704 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58c5f2-6381-402c-933a-4c89309be637 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:53.366934 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58c5f2-6381-402c-933a-4c89309be637 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:53.385315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58c5f2-6381-402c-933a-4c89309be637 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:53.388739 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58c5f2-6381-402c-933a-4c89309be637 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:53.406916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 936/1861] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:55.159899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f9251-fbf7-41a3-80b7-3cde82eab04c None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:55.160589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f9251-fbf7-41a3-80b7-3cde82eab04c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:55.160589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f9251-fbf7-41a3-80b7-3cde82eab04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:55.194870 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-848f9251-fbf7-41a3-80b7-3cde82eab04c None None] MFA Rules not processed for user `00fdea0388e34b1eb5bfff49a3de5b57`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:55.196662 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-848f9251-fbf7-41a3-80b7-3cde82eab04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:55.250519 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:55.333468 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 926/1862] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:55.365089 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58c5f2-6381-402c-933a-4c89309be637 service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:55.368394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-8d58c5f2-6381-402c-933a-4c89309be637 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:55.510389 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58c5f2-6381-402c-933a-4c89309be637 service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:55.517743 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfd6e7-cc88-4093-9c98-88ada0d3d03c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:55.518587 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfd6e7-cc88-4093-9c98-88ada0d3d03c service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:55.518937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfd6e7-cc88-4093-9c98-88ada0d3d03c service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:55.519335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfd6e7-cc88-4093-9c98-88ada0d3d03c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:55.540456 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfd6e7-cc88-4093-9c98-88ada0d3d03c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:55.543870 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfd6e7-cc88-4093-9c98-88ada0d3d03c service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:55.558178 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 937/1863] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:58.476717 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f9251-fbf7-41a3-80b7-3cde82eab04c None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:58.479076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-848f9251-fbf7-41a3-80b7-3cde82eab04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:58.495530 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f9251-fbf7-41a3-80b7-3cde82eab04c None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:58.500538 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9562147-4cfc-41f6-808f-ca78ee7f5918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:58.501274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9562147-4cfc-41f6-808f-ca78ee7f5918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:58.501645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9562147-4cfc-41f6-808f-ca78ee7f5918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:58.501985 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9562147-4cfc-41f6-808f-ca78ee7f5918 admin admin] PATH_INFO: `/v3/users/bbbd725b6b8b4089af57445c1df91337` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:58.503329 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9562147-4cfc-41f6-808f-ca78ee7f5918 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbbd725b6b8b4089af57445c1df91337)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:58.505346 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9562147-4cfc-41f6-808f-ca78ee7f5918 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:58.734696 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-d9562147-4cfc-41f6-808f-ca78ee7f5918 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbbd725b6b8b4089af57445c1df91337'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:58.853241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-d9562147-4cfc-41f6-808f-ca78ee7f5918 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbbd725b6b8b4089af57445c1df91337'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:58.853988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-d9562147-4cfc-41f6-808f-ca78ee7f5918 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbbd725b6b8b4089af57445c1df91337'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:58.867522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-d9562147-4cfc-41f6-808f-ca78ee7f5918 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbbd725b6b8b4089af57445c1df91337'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:58.868694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-d9562147-4cfc-41f6-808f-ca78ee7f5918 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbbd725b6b8b4089af57445c1df91337'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:52:58.896273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 927/1864] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:52:58 2022] DELETE /identity/v3/users/bbbd725b6b8b4089af57445c1df91337 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:52:58.903049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286c167-3bee-4d41-8b9e-88c3a137cf33 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:58.903810 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286c167-3bee-4d41-8b9e-88c3a137cf33 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:58.903810 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286c167-3bee-4d41-8b9e-88c3a137cf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:58.927187 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-f286c167-3bee-4d41-8b9e-88c3a137cf33 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:58.928852 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f286c167-3bee-4d41-8b9e-88c3a137cf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:59.011186 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 938/1865] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:59.021948 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9562147-4cfc-41f6-808f-ca78ee7f5918 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:59.024306 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d9562147-4cfc-41f6-808f-ca78ee7f5918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:59.094285 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9562147-4cfc-41f6-808f-ca78ee7f5918 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:59.098293 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792961c5-9a83-4b16-8fa5-eaf77a5ecc94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:59.098972 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792961c5-9a83-4b16-8fa5-eaf77a5ecc94 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:59.099297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792961c5-9a83-4b16-8fa5-eaf77a5ecc94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:59.099609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792961c5-9a83-4b16-8fa5-eaf77a5ecc94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:59.109925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792961c5-9a83-4b16-8fa5-eaf77a5ecc94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:59.111623 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792961c5-9a83-4b16-8fa5-eaf77a5ecc94 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:59.123221 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 928/1866] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:59.720585 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286c167-3bee-4d41-8b9e-88c3a137cf33 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:59.723474 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f286c167-3bee-4d41-8b9e-88c3a137cf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:59.747679 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286c167-3bee-4d41-8b9e-88c3a137cf33 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:59.759006 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8673ce-3081-4ed8-865f-0e62d6a5e9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:59.761080 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8673ce-3081-4ed8-865f-0e62d6a5e9d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:59.761411 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8673ce-3081-4ed8-865f-0e62d6a5e9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:59.764650 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8673ce-3081-4ed8-865f-0e62d6a5e9d6 admin admin] PATH_INFO: `/v3/projects/b6383a76b5b64fafb6caf61928850fc2` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:59.777760 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8673ce-3081-4ed8-865f-0e62d6a5e9d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6383a76b5b64fafb6caf61928850fc2)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:59.781216 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8673ce-3081-4ed8-865f-0e62d6a5e9d6 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:59.913778 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:59.943485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-7f8673ce-3081-4ed8-865f-0e62d6a5e9d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6383a76b5b64fafb6caf61928850fc2', 'request_id': 'req-7f8673ce-3081-4ed8-865f-0e62d6a5e9d6'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:00.017352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-7f8673ce-3081-4ed8-865f-0e62d6a5e9d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6383a76b5b64fafb6caf61928850fc2', 'request_id': 'req-7f8673ce-3081-4ed8-865f-0e62d6a5e9d6'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:00.032225 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-7f8673ce-3081-4ed8-865f-0e62d6a5e9d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6383a76b5b64fafb6caf61928850fc2', 'request_id': 'req-7f8673ce-3081-4ed8-865f-0e62d6a5e9d6'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:00.056719 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-7f8673ce-3081-4ed8-865f-0e62d6a5e9d6 admin admin] The token cache is being invalidate because project b6383a76b5b64fafb6caf61928850fc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:53:00.057012 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-7f8673ce-3081-4ed8-865f-0e62d6a5e9d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:00.057787 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-7f8673ce-3081-4ed8-865f-0e62d6a5e9d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:00.060285 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 939/1867] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:52:59 2022] DELETE /identity/v3/projects/b6383a76b5b64fafb6caf61928850fc2 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:00.065949 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792961c5-9a83-4b16-8fa5-eaf77a5ecc94 service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:00.089209 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792961c5-9a83-4b16-8fa5-eaf77a5ecc94 service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:00.150904 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:00.176690 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d07f4-1bf1-44ae-b230-a2472a7dcc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:00.177257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d07f4-1bf1-44ae-b230-a2472a7dcc4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:00.177654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d07f4-1bf1-44ae-b230-a2472a7dcc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:00.177931 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d07f4-1bf1-44ae-b230-a2472a7dcc4d admin admin] PATH_INFO: `/v3/users/004b6ad4901345b9a75e7c8d621e084f` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:00.179241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d07f4-1bf1-44ae-b230-a2472a7dcc4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=004b6ad4901345b9a75e7c8d621e084f)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:00.181772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d07f4-1bf1-44ae-b230-a2472a7dcc4d admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:00.405426 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-566d07f4-1bf1-44ae-b230-a2472a7dcc4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '004b6ad4901345b9a75e7c8d621e084f'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:00.484905 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-566d07f4-1bf1-44ae-b230-a2472a7dcc4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '004b6ad4901345b9a75e7c8d621e084f'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:00.485469 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-566d07f4-1bf1-44ae-b230-a2472a7dcc4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '004b6ad4901345b9a75e7c8d621e084f'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:00.495188 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-566d07f4-1bf1-44ae-b230-a2472a7dcc4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '004b6ad4901345b9a75e7c8d621e084f'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:00.495925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-566d07f4-1bf1-44ae-b230-a2472a7dcc4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '004b6ad4901345b9a75e7c8d621e084f'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:00.516029 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 929/1868] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:53:00 2022] DELETE /identity/v3/users/004b6ad4901345b9a75e7c8d621e084f => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:01.110192 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8673ce-3081-4ed8-865f-0e62d6a5e9d6 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:01.113409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-7f8673ce-3081-4ed8-865f-0e62d6a5e9d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:01.148996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8673ce-3081-4ed8-865f-0e62d6a5e9d6 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:01.320565 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabea7e0-e622-4e6c-93ff-24963f1ffbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:01.323731 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabea7e0-e622-4e6c-93ff-24963f1ffbed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:01.325231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabea7e0-e622-4e6c-93ff-24963f1ffbed admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:01.325231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabea7e0-e622-4e6c-93ff-24963f1ffbed admin admin] PATH_INFO: `/v3/projects/a4f226972b9d470f9370064cff6266d8` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:01.332100 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabea7e0-e622-4e6c-93ff-24963f1ffbed admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4f226972b9d470f9370064cff6266d8)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:01.335350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabea7e0-e622-4e6c-93ff-24963f1ffbed admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:01.514211 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-eabea7e0-e622-4e6c-93ff-24963f1ffbed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4f226972b9d470f9370064cff6266d8', 'request_id': 'req-eabea7e0-e622-4e6c-93ff-24963f1ffbed'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:01.589811 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-eabea7e0-e622-4e6c-93ff-24963f1ffbed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4f226972b9d470f9370064cff6266d8', 'request_id': 'req-eabea7e0-e622-4e6c-93ff-24963f1ffbed'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:01.602601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-eabea7e0-e622-4e6c-93ff-24963f1ffbed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4f226972b9d470f9370064cff6266d8', 'request_id': 'req-eabea7e0-e622-4e6c-93ff-24963f1ffbed'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:01.625948 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-eabea7e0-e622-4e6c-93ff-24963f1ffbed admin admin] The token cache is being invalidate because project a4f226972b9d470f9370064cff6266d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:53:01.625948 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-eabea7e0-e622-4e6c-93ff-24963f1ffbed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:01.626437 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-eabea7e0-e622-4e6c-93ff-24963f1ffbed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:01.628341 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 940/1869] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:53:01 2022] DELETE /identity/v3/projects/a4f226972b9d470f9370064cff6266d8 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:01.634901 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebf483-7904-4723-a557-3ae5e2af85ee None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:01.635732 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebf483-7904-4723-a557-3ae5e2af85ee None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:01.635732 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebf483-7904-4723-a557-3ae5e2af85ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:01.658720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-43ebf483-7904-4723-a557-3ae5e2af85ee None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:01.659978 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-43ebf483-7904-4723-a557-3ae5e2af85ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:01.746597 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 930/1870] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:53:01.763084 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0878e-a92c-462b-85dc-087eeed49b9f None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:01.763750 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0878e-a92c-462b-85dc-087eeed49b9f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:01.764318 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0878e-a92c-462b-85dc-087eeed49b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:01.799847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-86f0878e-a92c-462b-85dc-087eeed49b9f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:01.800012 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-86f0878e-a92c-462b-85dc-087eeed49b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:01.814272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 941/1871] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:01.820303 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebf483-7904-4723-a557-3ae5e2af85ee None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:01.838658 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebf483-7904-4723-a557-3ae5e2af85ee None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:01.874278 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22607bd8-c0eb-4ff7-b0c0-307235c37ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:01.874278 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22607bd8-c0eb-4ff7-b0c0-307235c37ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:01.874278 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22607bd8-c0eb-4ff7-b0c0-307235c37ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:01.874278 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22607bd8-c0eb-4ff7-b0c0-307235c37ac6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:01.875821 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22607bd8-c0eb-4ff7-b0c0-307235c37ac6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:53:01.875821 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22607bd8-c0eb-4ff7-b0c0-307235c37ac6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:01.875821 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22607bd8-c0eb-4ff7-b0c0-307235c37ac6 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:01.882892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 931/1872] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:53:01.889011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0878e-a92c-462b-85dc-087eeed49b9f None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:01.911568 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0878e-a92c-462b-85dc-087eeed49b9f None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:01.920976 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab7cbea-064d-438f-adee-94ea4faa988f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:01.922126 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7cbea-064d-438f-adee-94ea4faa988f admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:01.922665 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7cbea-064d-438f-adee-94ea4faa988f admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:01.923340 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7cbea-064d-438f-adee-94ea4faa988f admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:01.924345 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab7cbea-064d-438f-adee-94ea4faa988f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:01.927244 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab7cbea-064d-438f-adee-94ea4faa988f admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:02.062440 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e51623-c2da-42e5-9e40-dd74cba22bf5 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:02.062867 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e51623-c2da-42e5-9e40-dd74cba22bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:02.063383 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e51623-c2da-42e5-9e40-dd74cba22bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:02.098319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-34e51623-c2da-42e5-9e40-dd74cba22bf5 None None] MFA Rules not processed for user `00fdea0388e34b1eb5bfff49a3de5b57`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:02.100440 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-34e51623-c2da-42e5-9e40-dd74cba22bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:02.174839 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:02.188068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 932/1873] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:53:02.236451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 942/1874] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:53:01 2022] POST /identity/v3/projects => generated 379 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:02.243130 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e51623-c2da-42e5-9e40-dd74cba22bf5 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:02.295012 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e51623-c2da-42e5-9e40-dd74cba22bf5 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:02.410553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d536568-4030-4035-b4c1-2c958b1ca70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:02.411304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d536568-4030-4035-b4c1-2c958b1ca70f admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:02.411587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d536568-4030-4035-b4c1-2c958b1ca70f admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:02.412832 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d536568-4030-4035-b4c1-2c958b1ca70f admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:02.412832 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d536568-4030-4035-b4c1-2c958b1ca70f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:02.414682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d536568-4030-4035-b4c1-2c958b1ca70f admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:02.510241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 933/1875] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:53:02 2022] POST /identity/v3/users => generated 290 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:02.520999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab7cbea-064d-438f-adee-94ea4faa988f admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:02.559784 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab7cbea-064d-438f-adee-94ea4faa988f admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:02.574731 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec461c3-12f4-4769-a3bc-20240c4e221f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:02.577045 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec461c3-12f4-4769-a3bc-20240c4e221f admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:02.577507 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec461c3-12f4-4769-a3bc-20240c4e221f admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:02.578288 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec461c3-12f4-4769-a3bc-20240c4e221f admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:02.579569 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec461c3-12f4-4769-a3bc-20240c4e221f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:02.582774 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec461c3-12f4-4769-a3bc-20240c4e221f admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:02.610130 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 943/1876] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:02.616742 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d536568-4030-4035-b4c1-2c958b1ca70f admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:02.636212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d536568-4030-4035-b4c1-2c958b1ca70f admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:02.646830 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416233bb-2b85-4cec-800c-af5a5f77f0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:02.648500 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416233bb-2b85-4cec-800c-af5a5f77f0da admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:02.648975 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416233bb-2b85-4cec-800c-af5a5f77f0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:02.649490 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416233bb-2b85-4cec-800c-af5a5f77f0da admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:02.650812 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416233bb-2b85-4cec-800c-af5a5f77f0da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:02.654477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416233bb-2b85-4cec-800c-af5a5f77f0da admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:02.728844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 934/1877] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:02.730101 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec461c3-12f4-4769-a3bc-20240c4e221f admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:02.753576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec461c3-12f4-4769-a3bc-20240c4e221f admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:02.770541 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda182a-a576-42ba-bb71-5b7bbb24a556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:02.774543 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda182a-a576-42ba-bb71-5b7bbb24a556 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:02.775321 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda182a-a576-42ba-bb71-5b7bbb24a556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:02.776144 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda182a-a576-42ba-bb71-5b7bbb24a556 admin admin] PATH_INFO: `/v3/projects/ad0224d652f94927a7cb0ebe7ac3e09e/users/c37df9eb90934ebea6db9c049a80defb/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:02.790420 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:02.805875 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda182a-a576-42ba-bb71-5b7bbb24a556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad0224d652f94927a7cb0ebe7ac3e09e, user_id=c37df9eb90934ebea6db9c049a80defb, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:02.810002 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda182a-a576-42ba-bb71-5b7bbb24a556 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:02.941887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 944/1878] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:02 2022] PUT /identity/v3/projects/ad0224d652f94927a7cb0ebe7ac3e09e/users/c37df9eb90934ebea6db9c049a80defb/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:02.958070 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2a6a3-4a6a-476b-879e-d4c2dd4544bc None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:02.958800 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2a6a3-4a6a-476b-879e-d4c2dd4544bc None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:02.959883 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2a6a3-4a6a-476b-879e-d4c2dd4544bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:03.026660 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-e9d2a6a3-4a6a-476b-879e-d4c2dd4544bc None None] MFA Rules not processed for user `c37df9eb90934ebea6db9c049a80defb`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:03.026660 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-e9d2a6a3-4a6a-476b-879e-d4c2dd4544bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:03.110036 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 935/1879] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:03.117667 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda182a-a576-42ba-bb71-5b7bbb24a556 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:03.286403 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda182a-a576-42ba-bb71-5b7bbb24a556 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:03.300169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac954e-43b3-4085-91f9-b65861a725c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:03.301785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac954e-43b3-4085-91f9-b65861a725c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:03.302487 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac954e-43b3-4085-91f9-b65861a725c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:03.303793 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac954e-43b3-4085-91f9-b65861a725c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:03.305542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac954e-43b3-4085-91f9-b65861a725c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:03.311943 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac954e-43b3-4085-91f9-b65861a725c0 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:03.538938 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 945/1880] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:53:03 2022] POST /identity/v3/projects => generated 379 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:53:03.545865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d2a6a3-4a6a-476b-879e-d4c2dd4544bc None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:03.572834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d2a6a3-4a6a-476b-879e-d4c2dd4544bc None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:03.689304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f9df4-b1b1-4e55-b3e5-696f4be3cb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:03.690434 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f9df4-b1b1-4e55-b3e5-696f4be3cb11 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:03.690434 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f9df4-b1b1-4e55-b3e5-696f4be3cb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:03.691077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f9df4-b1b1-4e55-b3e5-696f4be3cb11 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:03.691706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f9df4-b1b1-4e55-b3e5-696f4be3cb11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:03.694698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f9df4-b1b1-4e55-b3e5-696f4be3cb11 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:03.705211 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:03.781397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 936/1881] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:53:03 2022] POST /identity/v3/users => generated 296 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:53:03.785705 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac954e-43b3-4085-91f9-b65861a725c0 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:03.817424 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac954e-43b3-4085-91f9-b65861a725c0 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:03.830654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb1c58-fd26-4912-9583-8f05cf9f045d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:03.831608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb1c58-fd26-4912-9583-8f05cf9f045d admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:03.832083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb1c58-fd26-4912-9583-8f05cf9f045d admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:03.832494 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb1c58-fd26-4912-9583-8f05cf9f045d admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:03.833498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb1c58-fd26-4912-9583-8f05cf9f045d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:03.836725 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb1c58-fd26-4912-9583-8f05cf9f045d admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:03.860991 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 946/1882] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:03.866664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f9df4-b1b1-4e55-b3e5-696f4be3cb11 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:03.887718 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f9df4-b1b1-4e55-b3e5-696f4be3cb11 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:03.897368 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fa62c-c146-473c-9b62-9d9488e6f618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:03.898163 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fa62c-c146-473c-9b62-9d9488e6f618 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:03.898652 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fa62c-c146-473c-9b62-9d9488e6f618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:03.900085 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fa62c-c146-473c-9b62-9d9488e6f618 admin admin] PATH_INFO: `/v3/projects/9374bedbc0ea4f9096e2043ddaa76057/users/655a80d71c7d4322b3c3cdd6e5748119/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:03.940656 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124fa62c-c146-473c-9b62-9d9488e6f618 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9374bedbc0ea4f9096e2043ddaa76057, user_id=655a80d71c7d4322b3c3cdd6e5748119, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:03.981255 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124fa62c-c146-473c-9b62-9d9488e6f618 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:04.176565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 937/1883] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:03 2022] PUT /identity/v3/projects/9374bedbc0ea4f9096e2043ddaa76057/users/655a80d71c7d4322b3c3cdd6e5748119/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:04.181619 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb1c58-fd26-4912-9583-8f05cf9f045d admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:04.217470 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb1c58-fd26-4912-9583-8f05cf9f045d admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:04.326681 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac697014-317b-44ec-b1c8-627a5e1873db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:04.327748 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac697014-317b-44ec-b1c8-627a5e1873db admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:04.327748 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac697014-317b-44ec-b1c8-627a5e1873db admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:04.328606 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac697014-317b-44ec-b1c8-627a5e1873db admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:04.330373 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac697014-317b-44ec-b1c8-627a5e1873db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:04.332740 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac697014-317b-44ec-b1c8-627a5e1873db admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:04.354935 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 947/1884] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:04.356540 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fa62c-c146-473c-9b62-9d9488e6f618 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:04.374274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fa62c-c146-473c-9b62-9d9488e6f618 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:04.382394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb8923-1f02-48e1-86f8-69ca2da67967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:04.383570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb8923-1f02-48e1-86f8-69ca2da67967 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:04.384268 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb8923-1f02-48e1-86f8-69ca2da67967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:04.384725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb8923-1f02-48e1-86f8-69ca2da67967 admin admin] PATH_INFO: `/v3/projects/9374bedbc0ea4f9096e2043ddaa76057/users/655a80d71c7d4322b3c3cdd6e5748119/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:04.387581 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb8923-1f02-48e1-86f8-69ca2da67967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9374bedbc0ea4f9096e2043ddaa76057, user_id=655a80d71c7d4322b3c3cdd6e5748119, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:04.391400 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb8923-1f02-48e1-86f8-69ca2da67967 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:04.425113 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f58f4-9b28-49a6-ac3e-b954f17d90c9 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:04.431451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f58f4-9b28-49a6-ac3e-b954f17d90c9 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:04.431451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f58f4-9b28-49a6-ac3e-b954f17d90c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:04.438705 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 938/1885] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:04 2022] PUT /identity/v3/projects/9374bedbc0ea4f9096e2043ddaa76057/users/655a80d71c7d4322b3c3cdd6e5748119/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:04.475776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-d33f58f4-9b28-49a6-ac3e-b954f17d90c9 None None] MFA Rules not processed for user `655a80d71c7d4322b3c3cdd6e5748119`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:04.478224 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-d33f58f4-9b28-49a6-ac3e-b954f17d90c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:04.659610 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:04.709842 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 948/1886] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:04 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:04.742213 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb8923-1f02-48e1-86f8-69ca2da67967 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:04.745785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-51fb8923-1f02-48e1-86f8-69ca2da67967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:04.821220 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb8923-1f02-48e1-86f8-69ca2da67967 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:04.825140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9633b81-27c5-4bb8-997b-bb0545dd9b8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:04.825785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9633b81-27c5-4bb8-997b-bb0545dd9b8e service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:04.826064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9633b81-27c5-4bb8-997b-bb0545dd9b8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:04.826333 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9633b81-27c5-4bb8-997b-bb0545dd9b8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:04.838831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9633b81-27c5-4bb8-997b-bb0545dd9b8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:04.840354 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9633b81-27c5-4bb8-997b-bb0545dd9b8e service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:04.854746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 939/1887] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:04 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:06.577311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f58f4-9b28-49a6-ac3e-b954f17d90c9 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:06.592377 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f58f4-9b28-49a6-ac3e-b954f17d90c9 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:06.598114 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518ea8c-940d-4e35-b031-7acfc72846f1 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '655a80d71c7d4322b3c3cdd6e5748119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9374bedbc0ea4f9096e2043ddaa76057', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:06.599070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518ea8c-940d-4e35-b031-7acfc72846f1 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:06.599539 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518ea8c-940d-4e35-b031-7acfc72846f1 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:06.601604 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518ea8c-940d-4e35-b031-7acfc72846f1 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:06.602604 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518ea8c-940d-4e35-b031-7acfc72846f1 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:53:06.603096 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518ea8c-940d-4e35-b031-7acfc72846f1 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:06.605888 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518ea8c-940d-4e35-b031-7acfc72846f1 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:06.629746 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 949/1888] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:53:06.635261 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9633b81-27c5-4bb8-997b-bb0545dd9b8e service cinder] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:06.648988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9633b81-27c5-4bb8-997b-bb0545dd9b8e service cinder] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:06.655285 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277f636-d6d6-4a7c-8b3e-a48644c21d08 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '655a80d71c7d4322b3c3cdd6e5748119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9374bedbc0ea4f9096e2043ddaa76057', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:06.668313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277f636-d6d6-4a7c-8b3e-a48644c21d08 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:06.668313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277f636-d6d6-4a7c-8b3e-a48644c21d08 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:06.668313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277f636-d6d6-4a7c-8b3e-a48644c21d08 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:06.668313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277f636-d6d6-4a7c-8b3e-a48644c21d08 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:06.668313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277f636-d6d6-4a7c-8b3e-a48644c21d08 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:06.845123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 940/1889] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:53:06 2022] POST /identity/v3/projects => generated 354 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:53:06.849534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518ea8c-940d-4e35-b031-7acfc72846f1 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:06.863809 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518ea8c-940d-4e35-b031-7acfc72846f1 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:06.949260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962de948-73df-47e6-8cf9-3fb7002288c1 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '655a80d71c7d4322b3c3cdd6e5748119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9374bedbc0ea4f9096e2043ddaa76057', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:06.950018 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962de948-73df-47e6-8cf9-3fb7002288c1 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:06.950370 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962de948-73df-47e6-8cf9-3fb7002288c1 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:06.950765 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962de948-73df-47e6-8cf9-3fb7002288c1 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:06.951774 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962de948-73df-47e6-8cf9-3fb7002288c1 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:53:06.952154 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962de948-73df-47e6-8cf9-3fb7002288c1 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:06.965869 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962de948-73df-47e6-8cf9-3fb7002288c1 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:07.015397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 950/1890] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:53:07.202052 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277f636-d6d6-4a7c-8b3e-a48644c21d08 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:07.239746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277f636-d6d6-4a7c-8b3e-a48644c21d08 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:07.243995 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962de948-73df-47e6-8cf9-3fb7002288c1 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:07.247102 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-962de948-73df-47e6-8cf9-3fb7002288c1 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:07.248620 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6955c-188e-4e32-9cce-a63024cc6197 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '655a80d71c7d4322b3c3cdd6e5748119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9374bedbc0ea4f9096e2043ddaa76057', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:07.250338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6955c-188e-4e32-9cce-a63024cc6197 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:07.251154 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6955c-188e-4e32-9cce-a63024cc6197 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:07.251851 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6955c-188e-4e32-9cce-a63024cc6197 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] PATH_INFO: `/v3/projects/290e72d1236541f7bc5fa0324ec5b77b` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:07.255605 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e6955c-188e-4e32-9cce-a63024cc6197 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] RBAC: Authorizing `identity:delete_project(project_id=290e72d1236541f7bc5fa0324ec5b77b)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:07.260169 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e6955c-188e-4e32-9cce-a63024cc6197 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:07.341153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962de948-73df-47e6-8cf9-3fb7002288c1 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:07.352258 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2f1b6-640f-46a0-9b47-ddb991a45a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:07.353989 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2f1b6-640f-46a0-9b47-ddb991a45a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:07.355189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2f1b6-640f-46a0-9b47-ddb991a45a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:07.355189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2f1b6-640f-46a0-9b47-ddb991a45a1a admin admin] PATH_INFO: `/v3/users/18428ac6efe14580b2422c9c2c335efe` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:07.371467 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2f1b6-640f-46a0-9b47-ddb991a45a1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=18428ac6efe14580b2422c9c2c335efe)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:07.375616 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2f1b6-640f-46a0-9b47-ddb991a45a1a admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:07.435576 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-a8e6955c-188e-4e32-9cce-a63024cc6197 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '290e72d1236541f7bc5fa0324ec5b77b', 'request_id': 'req-a8e6955c-188e-4e32-9cce-a63024cc6197'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:07.448925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-a8e6955c-188e-4e32-9cce-a63024cc6197 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '290e72d1236541f7bc5fa0324ec5b77b', 'request_id': 'req-a8e6955c-188e-4e32-9cce-a63024cc6197'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:07.462849 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-a8e6955c-188e-4e32-9cce-a63024cc6197 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '290e72d1236541f7bc5fa0324ec5b77b', 'request_id': 'req-a8e6955c-188e-4e32-9cce-a63024cc6197'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:07.609380 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-a8e6955c-188e-4e32-9cce-a63024cc6197 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] The token cache is being invalidate because project 290e72d1236541f7bc5fa0324ec5b77b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:53:07.609921 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-a8e6955c-188e-4e32-9cce-a63024cc6197 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:07.611075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-a8e6955c-188e-4e32-9cce-a63024cc6197 tempest-VolumeQuotaClassesTest-452737649 tempest-VolumeQuotaClassesTest-452737649-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:07.612884 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 941/1891] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:53:07 2022] DELETE /identity/v3/projects/290e72d1236541f7bc5fa0324ec5b77b => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:07.650350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0bb2f1b6-640f-46a0-9b47-ddb991a45a1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18428ac6efe14580b2422c9c2c335efe'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:07.738018 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0bb2f1b6-640f-46a0-9b47-ddb991a45a1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18428ac6efe14580b2422c9c2c335efe'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:07.739317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0bb2f1b6-640f-46a0-9b47-ddb991a45a1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18428ac6efe14580b2422c9c2c335efe'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:07.757193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0bb2f1b6-640f-46a0-9b47-ddb991a45a1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18428ac6efe14580b2422c9c2c335efe'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:07.759648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:07.784184 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0bb2f1b6-640f-46a0-9b47-ddb991a45a1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18428ac6efe14580b2422c9c2c335efe'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:07.793813 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 951/1892] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:53:07 2022] DELETE /identity/v3/users/18428ac6efe14580b2422c9c2c335efe => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:07.798982 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17fc11-e76a-40af-a713-17a31b33c4f2 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:07.800187 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17fc11-e76a-40af-a713-17a31b33c4f2 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:07.804665 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17fc11-e76a-40af-a713-17a31b33c4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:07.833786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-1b17fc11-e76a-40af-a713-17a31b33c4f2 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:07.837451 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-1b17fc11-e76a-40af-a713-17a31b33c4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:07.869018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:INSERT=6,keystone:SELECT=87,keystone:DELETE=7 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:07.949662 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 942/1893] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:07.967041 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2f1b6-640f-46a0-9b47-ddb991a45a1a admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:07.969175 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-0bb2f1b6-640f-46a0-9b47-ddb991a45a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:08.089803 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2f1b6-640f-46a0-9b47-ddb991a45a1a admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:08.096523 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538d4ba-a9c8-48b0-a8b5-4ffa51d84619 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:08.098387 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538d4ba-a9c8-48b0-a8b5-4ffa51d84619 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:08.098387 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538d4ba-a9c8-48b0-a8b5-4ffa51d84619 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:08.099077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538d4ba-a9c8-48b0-a8b5-4ffa51d84619 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:08.121912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538d4ba-a9c8-48b0-a8b5-4ffa51d84619 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:08.125540 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538d4ba-a9c8-48b0-a8b5-4ffa51d84619 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:08.146089 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 952/1894] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:08.371330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17fc11-e76a-40af-a713-17a31b33c4f2 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:08.372826 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-1b17fc11-e76a-40af-a713-17a31b33c4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:08.400230 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17fc11-e76a-40af-a713-17a31b33c4f2 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:08.417722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0650e-2dac-420b-87ce-f3669236689d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:08.419707 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0650e-2dac-420b-87ce-f3669236689d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:08.420172 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0650e-2dac-420b-87ce-f3669236689d admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:08.423050 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0650e-2dac-420b-87ce-f3669236689d admin admin] PATH_INFO: `/v3/users/c37df9eb90934ebea6db9c049a80defb` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:08.424959 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e0650e-2dac-420b-87ce-f3669236689d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c37df9eb90934ebea6db9c049a80defb)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:08.445516 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e0650e-2dac-420b-87ce-f3669236689d admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:08.833876 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-62e0650e-2dac-420b-87ce-f3669236689d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c37df9eb90934ebea6db9c049a80defb'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:08.940322 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-62e0650e-2dac-420b-87ce-f3669236689d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c37df9eb90934ebea6db9c049a80defb'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:08.949209 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-62e0650e-2dac-420b-87ce-f3669236689d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c37df9eb90934ebea6db9c049a80defb'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:08.979286 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-62e0650e-2dac-420b-87ce-f3669236689d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c37df9eb90934ebea6db9c049a80defb'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:08.981208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-62e0650e-2dac-420b-87ce-f3669236689d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c37df9eb90934ebea6db9c049a80defb'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:09.093925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 943/1895] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:53:08 2022] DELETE /identity/v3/users/c37df9eb90934ebea6db9c049a80defb => generated 0 bytes in 719 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:09.096255 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e5d07-5757-47f9-9cee-0df86f95d4cf None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:09.097123 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e5d07-5757-47f9-9cee-0df86f95d4cf None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:09.097123 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e5d07-5757-47f9-9cee-0df86f95d4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:09.146058 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-928e5d07-5757-47f9-9cee-0df86f95d4cf None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:09.148077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-928e5d07-5757-47f9-9cee-0df86f95d4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:09.260897 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 953/1896] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:09.267777 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0650e-2dac-420b-87ce-f3669236689d admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:09.269471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-62e0650e-2dac-420b-87ce-f3669236689d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:09.271300 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e5d07-5757-47f9-9cee-0df86f95d4cf None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:09.276388 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-928e5d07-5757-47f9-9cee-0df86f95d4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:09.292846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0650e-2dac-420b-87ce-f3669236689d admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:09.298647 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4add61-346d-4c96-bb7e-b49933692bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:09.299459 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4add61-346d-4c96-bb7e-b49933692bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:09.299740 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4add61-346d-4c96-bb7e-b49933692bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:09.300046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4add61-346d-4c96-bb7e-b49933692bb6 admin admin] PATH_INFO: `/v3/projects/4cb0de2178b344c3a941e04b29a3a06e` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:09.301531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4add61-346d-4c96-bb7e-b49933692bb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cb0de2178b344c3a941e04b29a3a06e)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:09.304108 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4add61-346d-4c96-bb7e-b49933692bb6 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:09.431159 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e5d07-5757-47f9-9cee-0df86f95d4cf None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:09.453391 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dac805-595e-4883-accf-768847b42133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:09.454725 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dac805-595e-4883-accf-768847b42133 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:09.455200 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dac805-595e-4883-accf-768847b42133 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:09.456621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dac805-595e-4883-accf-768847b42133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:09.488059 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dac805-595e-4883-accf-768847b42133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:09.490917 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dac805-595e-4883-accf-768847b42133 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:09.527121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-3d4add61-346d-4c96-bb7e-b49933692bb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cb0de2178b344c3a941e04b29a3a06e', 'request_id': 'req-3d4add61-346d-4c96-bb7e-b49933692bb6'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:09.534153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 954/1897] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:09.546543 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-3d4add61-346d-4c96-bb7e-b49933692bb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cb0de2178b344c3a941e04b29a3a06e', 'request_id': 'req-3d4add61-346d-4c96-bb7e-b49933692bb6'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:09.565936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-3d4add61-346d-4c96-bb7e-b49933692bb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cb0de2178b344c3a941e04b29a3a06e', 'request_id': 'req-3d4add61-346d-4c96-bb7e-b49933692bb6'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:09.698913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-3d4add61-346d-4c96-bb7e-b49933692bb6 admin admin] The token cache is being invalidate because project 4cb0de2178b344c3a941e04b29a3a06e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:53:09.700199 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-3d4add61-346d-4c96-bb7e-b49933692bb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:09.705885 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-3d4add61-346d-4c96-bb7e-b49933692bb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:09.712824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 944/1898] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:53:09 2022] DELETE /identity/v3/projects/4cb0de2178b344c3a941e04b29a3a06e => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:09.718340 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dac805-595e-4883-accf-768847b42133 service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:09.744793 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dac805-595e-4883-accf-768847b42133 service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:09.801491 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:09.869141 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534d718-cf15-46b5-97d3-fbfc20c01f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:09.869452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534d718-cf15-46b5-97d3-fbfc20c01f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:09.869762 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534d718-cf15-46b5-97d3-fbfc20c01f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:09.869910 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534d718-cf15-46b5-97d3-fbfc20c01f24 admin admin] PATH_INFO: `/v3/users/b363a8009629476baf4418146e1ebe7f` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:09.871787 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534d718-cf15-46b5-97d3-fbfc20c01f24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b363a8009629476baf4418146e1ebe7f)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:09.873770 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534d718-cf15-46b5-97d3-fbfc20c01f24 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:10.343712 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-a534d718-cf15-46b5-97d3-fbfc20c01f24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b363a8009629476baf4418146e1ebe7f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:10.555196 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-a534d718-cf15-46b5-97d3-fbfc20c01f24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b363a8009629476baf4418146e1ebe7f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:10.560364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-a534d718-cf15-46b5-97d3-fbfc20c01f24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b363a8009629476baf4418146e1ebe7f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:10.592074 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-a534d718-cf15-46b5-97d3-fbfc20c01f24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b363a8009629476baf4418146e1ebe7f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:10.633883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-a534d718-cf15-46b5-97d3-fbfc20c01f24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b363a8009629476baf4418146e1ebe7f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:10.648647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 955/1899] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:53:09 2022] DELETE /identity/v3/users/b363a8009629476baf4418146e1ebe7f => generated 0 bytes in 931 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:10.764942 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4add61-346d-4c96-bb7e-b49933692bb6 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:10.769851 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-3d4add61-346d-4c96-bb7e-b49933692bb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:10.799006 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4add61-346d-4c96-bb7e-b49933692bb6 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:10.823781 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:10.928489 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15704ce2-da0d-46c2-87ca-fe7d7ab1a63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:10.930034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15704ce2-da0d-46c2-87ca-fe7d7ab1a63e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:10.930539 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15704ce2-da0d-46c2-87ca-fe7d7ab1a63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:10.931357 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15704ce2-da0d-46c2-87ca-fe7d7ab1a63e admin admin] PATH_INFO: `/v3/projects/ad0224d652f94927a7cb0ebe7ac3e09e` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:10.934204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15704ce2-da0d-46c2-87ca-fe7d7ab1a63e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad0224d652f94927a7cb0ebe7ac3e09e)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:10.939807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15704ce2-da0d-46c2-87ca-fe7d7ab1a63e admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:11.138755 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-15704ce2-da0d-46c2-87ca-fe7d7ab1a63e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad0224d652f94927a7cb0ebe7ac3e09e', 'request_id': 'req-15704ce2-da0d-46c2-87ca-fe7d7ab1a63e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:11.151285 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-15704ce2-da0d-46c2-87ca-fe7d7ab1a63e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad0224d652f94927a7cb0ebe7ac3e09e', 'request_id': 'req-15704ce2-da0d-46c2-87ca-fe7d7ab1a63e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:11.158810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-15704ce2-da0d-46c2-87ca-fe7d7ab1a63e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad0224d652f94927a7cb0ebe7ac3e09e', 'request_id': 'req-15704ce2-da0d-46c2-87ca-fe7d7ab1a63e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:11.318740 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-15704ce2-da0d-46c2-87ca-fe7d7ab1a63e admin admin] The token cache is being invalidate because project ad0224d652f94927a7cb0ebe7ac3e09e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:53:11.319039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-15704ce2-da0d-46c2-87ca-fe7d7ab1a63e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:11.319679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-15704ce2-da0d-46c2-87ca-fe7d7ab1a63e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:11.321291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 945/1900] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:53:10 2022] DELETE /identity/v3/projects/ad0224d652f94927a7cb0ebe7ac3e09e => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:11.325948 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534d718-cf15-46b5-97d3-fbfc20c01f24 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:11.327676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-a534d718-cf15-46b5-97d3-fbfc20c01f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:11.342018 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534d718-cf15-46b5-97d3-fbfc20c01f24 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:11.417577 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74747b-d69b-46ca-ac2e-f8910bc4046d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:11.418773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74747b-d69b-46ca-ac2e-f8910bc4046d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:11.418773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74747b-d69b-46ca-ac2e-f8910bc4046d admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:11.419063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74747b-d69b-46ca-ac2e-f8910bc4046d admin admin] PATH_INFO: `/v3/users/655a80d71c7d4322b3c3cdd6e5748119` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:11.421223 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74747b-d69b-46ca-ac2e-f8910bc4046d admin admin] RBAC: Authorizing `identity:delete_user(user_id=655a80d71c7d4322b3c3cdd6e5748119)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:11.424057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74747b-d69b-46ca-ac2e-f8910bc4046d admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:11.493946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15704ce2-da0d-46c2-87ca-fe7d7ab1a63e admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:11.495949 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-15704ce2-da0d-46c2-87ca-fe7d7ab1a63e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:11.523282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15704ce2-da0d-46c2-87ca-fe7d7ab1a63e admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:11.543911 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403cb3c5-1252-4eb3-bfa4-6a800a19996a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:11.544207 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403cb3c5-1252-4eb3-bfa4-6a800a19996a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:11.544460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403cb3c5-1252-4eb3-bfa4-6a800a19996a admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:11.544738 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403cb3c5-1252-4eb3-bfa4-6a800a19996a admin admin] PATH_INFO: `/v3/projects/1505f06d269d41b3b899f072ba37bfc9` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:11.546482 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403cb3c5-1252-4eb3-bfa4-6a800a19996a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1505f06d269d41b3b899f072ba37bfc9)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:11.551929 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403cb3c5-1252-4eb3-bfa4-6a800a19996a admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:11.905606 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3a74747b-d69b-46ca-ac2e-f8910bc4046d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '655a80d71c7d4322b3c3cdd6e5748119'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:11.909813 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:11.920013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-403cb3c5-1252-4eb3-bfa4-6a800a19996a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1505f06d269d41b3b899f072ba37bfc9', 'request_id': 'req-403cb3c5-1252-4eb3-bfa4-6a800a19996a'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:11.935563 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-403cb3c5-1252-4eb3-bfa4-6a800a19996a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1505f06d269d41b3b899f072ba37bfc9', 'request_id': 'req-403cb3c5-1252-4eb3-bfa4-6a800a19996a'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:11.949338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-403cb3c5-1252-4eb3-bfa4-6a800a19996a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1505f06d269d41b3b899f072ba37bfc9', 'request_id': 'req-403cb3c5-1252-4eb3-bfa4-6a800a19996a'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:11.984406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3a74747b-d69b-46ca-ac2e-f8910bc4046d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '655a80d71c7d4322b3c3cdd6e5748119'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:11.985256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3a74747b-d69b-46ca-ac2e-f8910bc4046d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '655a80d71c7d4322b3c3cdd6e5748119'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:12.006303 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3a74747b-d69b-46ca-ac2e-f8910bc4046d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '655a80d71c7d4322b3c3cdd6e5748119'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:12.032329 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3a74747b-d69b-46ca-ac2e-f8910bc4046d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '655a80d71c7d4322b3c3cdd6e5748119'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:12.057388 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 956/1901] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:53:11 2022] DELETE /identity/v3/users/655a80d71c7d4322b3c3cdd6e5748119 => generated 0 bytes in 732 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:12.137735 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-403cb3c5-1252-4eb3-bfa4-6a800a19996a admin admin] The token cache is being invalidate because project 1505f06d269d41b3b899f072ba37bfc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:53:12.138819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-403cb3c5-1252-4eb3-bfa4-6a800a19996a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:12.148219 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-403cb3c5-1252-4eb3-bfa4-6a800a19996a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:12.159660 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712edf74-ab0f-4511-a440-70ce1f449c23 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:12.160159 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712edf74-ab0f-4511-a440-70ce1f449c23 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:12.160611 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712edf74-ab0f-4511-a440-70ce1f449c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:12.165467 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 946/1902] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:53:11 2022] DELETE /identity/v3/projects/1505f06d269d41b3b899f072ba37bfc9 => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:12.198064 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-712edf74-ab0f-4511-a440-70ce1f449c23 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:12.200575 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-712edf74-ab0f-4511-a440-70ce1f449c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:12.349981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 957/1903] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:12.355699 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6f139-a1dc-44af-918a-84cb8714419f None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:12.356315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6f139-a1dc-44af-918a-84cb8714419f None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:12.356632 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6f139-a1dc-44af-918a-84cb8714419f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:12.383959 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-29c6f139-a1dc-44af-918a-84cb8714419f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:12.385230 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-29c6f139-a1dc-44af-918a-84cb8714419f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:12.394075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 947/1904] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:12.405190 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712edf74-ab0f-4511-a440-70ce1f449c23 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:12.417143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712edf74-ab0f-4511-a440-70ce1f449c23 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:12.421960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed183de1-e3a8-41a2-b144-2ef52f98ab16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:12.422680 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed183de1-e3a8-41a2-b144-2ef52f98ab16 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:12.423117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed183de1-e3a8-41a2-b144-2ef52f98ab16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:12.423395 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed183de1-e3a8-41a2-b144-2ef52f98ab16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:12.424058 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed183de1-e3a8-41a2-b144-2ef52f98ab16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:53:12.424360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed183de1-e3a8-41a2-b144-2ef52f98ab16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:12.426026 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed183de1-e3a8-41a2-b144-2ef52f98ab16 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:12.447679 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 958/1905] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:53:12.452856 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6f139-a1dc-44af-918a-84cb8714419f None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:12.465059 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6f139-a1dc-44af-918a-84cb8714419f None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:12.470627 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7afd9-7591-41f7-b9bb-13063056b161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:12.471436 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7afd9-7591-41f7-b9bb-13063056b161 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:12.471824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7afd9-7591-41f7-b9bb-13063056b161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:12.472195 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7afd9-7591-41f7-b9bb-13063056b161 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:12.472964 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7afd9-7591-41f7-b9bb-13063056b161 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:12.475636 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7afd9-7591-41f7-b9bb-13063056b161 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:12.657416 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:12.674239 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 948/1906] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:53:12 2022] POST /identity/v3/projects => generated 393 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:53:12.681021 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed183de1-e3a8-41a2-b144-2ef52f98ab16 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:12.703604 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed183de1-e3a8-41a2-b144-2ef52f98ab16 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:12.814380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685a695-3f32-4789-a508-e56893ff00e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:12.815108 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685a695-3f32-4789-a508-e56893ff00e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:12.815383 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685a695-3f32-4789-a508-e56893ff00e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:12.815656 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685a695-3f32-4789-a508-e56893ff00e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:12.816292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685a695-3f32-4789-a508-e56893ff00e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:12.819726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685a695-3f32-4789-a508-e56893ff00e5 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:12.852916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7afd9-7591-41f7-b9bb-13063056b161 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:12.856315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-46e7afd9-7591-41f7-b9bb-13063056b161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:12.885061 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7afd9-7591-41f7-b9bb-13063056b161 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:12.897707 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc3ed6-4881-435b-a083-f1ed70ce5f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:12.899405 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc3ed6-4881-435b-a083-f1ed70ce5f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:12.901676 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc3ed6-4881-435b-a083-f1ed70ce5f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:12.901676 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc3ed6-4881-435b-a083-f1ed70ce5f8a admin admin] PATH_INFO: `/v3/projects/9374bedbc0ea4f9096e2043ddaa76057` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:12.903270 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffc3ed6-4881-435b-a083-f1ed70ce5f8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9374bedbc0ea4f9096e2043ddaa76057)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:12.907532 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffc3ed6-4881-435b-a083-f1ed70ce5f8a admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:12.939464 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 959/1907] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:53:12 2022] POST /identity/v3/users => generated 297 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:12.948916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685a695-3f32-4789-a508-e56893ff00e5 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:12.983143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685a695-3f32-4789-a508-e56893ff00e5 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:13.002344 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da938c13-46ce-4a9a-9f79-a94469039528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:13.003121 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da938c13-46ce-4a9a-9f79-a94469039528 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:13.003388 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da938c13-46ce-4a9a-9f79-a94469039528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:13.003859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da938c13-46ce-4a9a-9f79-a94469039528 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:13.007265 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da938c13-46ce-4a9a-9f79-a94469039528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:13.009591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da938c13-46ce-4a9a-9f79-a94469039528 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:13.039148 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 960/1908] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:13.042048 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da938c13-46ce-4a9a-9f79-a94469039528 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:13.077024 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da938c13-46ce-4a9a-9f79-a94469039528 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:13.101991 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d497e321-8b52-47c4-9fba-8d980737f15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:13.101991 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497e321-8b52-47c4-9fba-8d980737f15b admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:13.101991 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497e321-8b52-47c4-9fba-8d980737f15b admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:13.101991 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497e321-8b52-47c4-9fba-8d980737f15b admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:13.106160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d497e321-8b52-47c4-9fba-8d980737f15b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:13.111169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d497e321-8b52-47c4-9fba-8d980737f15b admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:13.122746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-5ffc3ed6-4881-435b-a083-f1ed70ce5f8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9374bedbc0ea4f9096e2043ddaa76057', 'request_id': 'req-5ffc3ed6-4881-435b-a083-f1ed70ce5f8a'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:13.131515 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-5ffc3ed6-4881-435b-a083-f1ed70ce5f8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9374bedbc0ea4f9096e2043ddaa76057', 'request_id': 'req-5ffc3ed6-4881-435b-a083-f1ed70ce5f8a'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:13.135614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 961/1909] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:13.137108 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d497e321-8b52-47c4-9fba-8d980737f15b admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:13.147574 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-5ffc3ed6-4881-435b-a083-f1ed70ce5f8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9374bedbc0ea4f9096e2043ddaa76057', 'request_id': 'req-5ffc3ed6-4881-435b-a083-f1ed70ce5f8a'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:13.159321 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d497e321-8b52-47c4-9fba-8d980737f15b admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:13.263989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-5ffc3ed6-4881-435b-a083-f1ed70ce5f8a admin admin] The token cache is being invalidate because project 9374bedbc0ea4f9096e2043ddaa76057 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:53:13.264269 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-5ffc3ed6-4881-435b-a083-f1ed70ce5f8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:13.265167 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-5ffc3ed6-4881-435b-a083-f1ed70ce5f8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:13.270914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef061b48-3fcd-487e-9af6-19bed543b6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:13.272900 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef061b48-3fcd-487e-9af6-19bed543b6ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:13.273252 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef061b48-3fcd-487e-9af6-19bed543b6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:13.273595 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef061b48-3fcd-487e-9af6-19bed543b6ea admin admin] PATH_INFO: `/v3/projects/605580fa86e84d289ae3f88a6f4a3ded/users/8effb799317c4e71bd68c977e30b4d29/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:13.273743 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 949/1910] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:53:12 2022] DELETE /identity/v3/projects/9374bedbc0ea4f9096e2043ddaa76057 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:13.275267 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de7d26-f7d9-4fb7-b93a-d090b2f1da1e None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:13.275555 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de7d26-f7d9-4fb7-b93a-d090b2f1da1e None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:13.275828 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de7d26-f7d9-4fb7-b93a-d090b2f1da1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:13.293614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef061b48-3fcd-487e-9af6-19bed543b6ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=605580fa86e84d289ae3f88a6f4a3ded, user_id=8effb799317c4e71bd68c977e30b4d29, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:13.296692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef061b48-3fcd-487e-9af6-19bed543b6ea admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:13.301762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-b5de7d26-f7d9-4fb7-b93a-d090b2f1da1e None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:13.303713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-b5de7d26-f7d9-4fb7-b93a-d090b2f1da1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:13.314170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 950/1911] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:13.330729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87da4a4-e2f8-4cd2-bb87-a11f75639659 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:13.330729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87da4a4-e2f8-4cd2-bb87-a11f75639659 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:13.330729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87da4a4-e2f8-4cd2-bb87-a11f75639659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:13.348553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-d87da4a4-e2f8-4cd2-bb87-a11f75639659 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:13.351230 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d87da4a4-e2f8-4cd2-bb87-a11f75639659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:13.361520 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 951/1912] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:13.367244 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87da4a4-e2f8-4cd2-bb87-a11f75639659 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:13.377795 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87da4a4-e2f8-4cd2-bb87-a11f75639659 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:13.386540 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2820858-9d54-4489-9731-dbd49a4ba830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:13.393480 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2820858-9d54-4489-9731-dbd49a4ba830 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:13.393480 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2820858-9d54-4489-9731-dbd49a4ba830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:13.393480 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2820858-9d54-4489-9731-dbd49a4ba830 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:13.393480 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2820858-9d54-4489-9731-dbd49a4ba830 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:53:13.393480 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2820858-9d54-4489-9731-dbd49a4ba830 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:13.394890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2820858-9d54-4489-9731-dbd49a4ba830 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:13.412873 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 952/1913] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:53:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:53:13.417689 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2820858-9d54-4489-9731-dbd49a4ba830 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:13.435287 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 962/1914] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:13 2022] PUT /identity/v3/projects/605580fa86e84d289ae3f88a6f4a3ded/users/8effb799317c4e71bd68c977e30b4d29/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:13.437553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2820858-9d54-4489-9731-dbd49a4ba830 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:13.454282 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e751114-3569-4a0b-98af-9b35032ced2b None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:13.454737 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e751114-3569-4a0b-98af-9b35032ced2b None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:13.460568 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e751114-3569-4a0b-98af-9b35032ced2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:13.500126 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-6e751114-3569-4a0b-98af-9b35032ced2b None None] MFA Rules not processed for user `8effb799317c4e71bd68c977e30b4d29`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:13.502351 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-6e751114-3569-4a0b-98af-9b35032ced2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:13.572767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae5ae1-a866-4419-a8ee-c2a91bef1b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:13.574016 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae5ae1-a866-4419-a8ee-c2a91bef1b12 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:13.574693 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae5ae1-a866-4419-a8ee-c2a91bef1b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:13.575468 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae5ae1-a866-4419-a8ee-c2a91bef1b12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:13.576748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae5ae1-a866-4419-a8ee-c2a91bef1b12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:13.581183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae5ae1-a866-4419-a8ee-c2a91bef1b12 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:13.620351 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:13.638348 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 963/1915] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:13.645726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e751114-3569-4a0b-98af-9b35032ced2b None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:13.649980 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-6e751114-3569-4a0b-98af-9b35032ced2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:13.669975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e751114-3569-4a0b-98af-9b35032ced2b None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:13.677417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbddf7-04b7-4c83-9cc1-74e8659c8b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:13.678324 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbddf7-04b7-4c83-9cc1-74e8659c8b0e admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:13.678771 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbddf7-04b7-4c83-9cc1-74e8659c8b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:13.679264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbddf7-04b7-4c83-9cc1-74e8659c8b0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:13.680439 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cbddf7-04b7-4c83-9cc1-74e8659c8b0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:13.684390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cbddf7-04b7-4c83-9cc1-74e8659c8b0e admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:13.764253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 953/1916] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:53:13 2022] POST /identity/v3/projects => generated 375 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:13.768659 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae5ae1-a866-4419-a8ee-c2a91bef1b12 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:13.844759 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 964/1917] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:53:13 2022] POST /identity/v3/projects => generated 393 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:13.850406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbddf7-04b7-4c83-9cc1-74e8659c8b0e admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:13.863863 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae5ae1-a866-4419-a8ee-c2a91bef1b12 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:13.960822 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbddf7-04b7-4c83-9cc1-74e8659c8b0e admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:13.967168 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422c5ef-dd47-42f3-bdb2-42efd5b08c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:13.967881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422c5ef-dd47-42f3-bdb2-42efd5b08c18 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:13.968159 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422c5ef-dd47-42f3-bdb2-42efd5b08c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:13.968426 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422c5ef-dd47-42f3-bdb2-42efd5b08c18 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:13.969070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422c5ef-dd47-42f3-bdb2-42efd5b08c18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:13.971237 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422c5ef-dd47-42f3-bdb2-42efd5b08c18 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:13.982166 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2d5d1-d3a4-4610-9e1c-6f1b23b83320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:13.982825 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2d5d1-d3a4-4610-9e1c-6f1b23b83320 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:13.983314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2d5d1-d3a4-4610-9e1c-6f1b23b83320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:13.983589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2d5d1-d3a4-4610-9e1c-6f1b23b83320 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:13.984242 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc2d5d1-d3a4-4610-9e1c-6f1b23b83320 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:13.986267 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc2d5d1-d3a4-4610-9e1c-6f1b23b83320 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:14.090558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 965/1918] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:53:13 2022] POST /identity/v3/users => generated 303 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:14.094805 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422c5ef-dd47-42f3-bdb2-42efd5b08c18 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:14.095114 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 954/1919] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:53:13 2022] POST /identity/v3/users => generated 288 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:53:14.101809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2d5d1-d3a4-4610-9e1c-6f1b23b83320 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:14.110001 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422c5ef-dd47-42f3-bdb2-42efd5b08c18 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:14.122559 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11b4b8-78d7-41b5-9eb4-6184077edc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:14.123627 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11b4b8-78d7-41b5-9eb4-6184077edc5e admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:14.124593 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11b4b8-78d7-41b5-9eb4-6184077edc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:14.124937 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2d5d1-d3a4-4610-9e1c-6f1b23b83320 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:14.125395 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11b4b8-78d7-41b5-9eb4-6184077edc5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:14.126327 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11b4b8-78d7-41b5-9eb4-6184077edc5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:14.129773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11b4b8-78d7-41b5-9eb4-6184077edc5e admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:14.148029 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea5aec-2202-41b2-b7bd-3e3cb761bc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:14.160734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea5aec-2202-41b2-b7bd-3e3cb761bc70 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:14.160734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea5aec-2202-41b2-b7bd-3e3cb761bc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:14.160734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea5aec-2202-41b2-b7bd-3e3cb761bc70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:14.160734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea5aec-2202-41b2-b7bd-3e3cb761bc70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:14.161124 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 966/1920] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:53:14.161213 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea5aec-2202-41b2-b7bd-3e3cb761bc70 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:14.164707 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11b4b8-78d7-41b5-9eb4-6184077edc5e admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:14.168839 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 955/1921] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:14.171706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea5aec-2202-41b2-b7bd-3e3cb761bc70 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:14.184009 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11b4b8-78d7-41b5-9eb4-6184077edc5e admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:14.187630 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea5aec-2202-41b2-b7bd-3e3cb761bc70 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:14.189948 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf89d57-7ca8-498b-a24a-c07e5216a680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:14.194686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf89d57-7ca8-498b-a24a-c07e5216a680 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:14.194686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf89d57-7ca8-498b-a24a-c07e5216a680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:14.194686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf89d57-7ca8-498b-a24a-c07e5216a680 admin admin] PATH_INFO: `/v3/projects/149ee4cf8d20400c8532289d783ea636/users/d492d8c2448e4248917a0ea3e3b05bd8/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:14.195574 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a1ec2-8944-466c-8b2c-8260cbffd4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:14.196531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a1ec2-8944-466c-8b2c-8260cbffd4b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:14.196722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a1ec2-8944-466c-8b2c-8260cbffd4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:14.197009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a1ec2-8944-466c-8b2c-8260cbffd4b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:14.197658 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a1ec2-8944-466c-8b2c-8260cbffd4b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:14.199356 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a1ec2-8944-466c-8b2c-8260cbffd4b7 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:14.211763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf89d57-7ca8-498b-a24a-c07e5216a680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=149ee4cf8d20400c8532289d783ea636, user_id=d492d8c2448e4248917a0ea3e3b05bd8, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:14.218388 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 956/1922] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:53:14.218682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf89d57-7ca8-498b-a24a-c07e5216a680 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:14.219648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a1ec2-8944-466c-8b2c-8260cbffd4b7 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:14.237450 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a1ec2-8944-466c-8b2c-8260cbffd4b7 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:14.246019 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f5d14-252e-43a8-88f1-4ccc0dfc0660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:14.247419 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f5d14-252e-43a8-88f1-4ccc0dfc0660 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:14.247865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f5d14-252e-43a8-88f1-4ccc0dfc0660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:14.248389 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f5d14-252e-43a8-88f1-4ccc0dfc0660 admin admin] PATH_INFO: `/v3/projects/f0d903adbca34fd4b3c1f1e3f838fb7c/users/ff230e7efdcd405e8be7034d567357e1/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:14.296211 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4f5d14-252e-43a8-88f1-4ccc0dfc0660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0d903adbca34fd4b3c1f1e3f838fb7c, user_id=ff230e7efdcd405e8be7034d567357e1, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:14.301976 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4f5d14-252e-43a8-88f1-4ccc0dfc0660 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:14.326454 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 967/1923] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:14 2022] PUT /identity/v3/projects/149ee4cf8d20400c8532289d783ea636/users/d492d8c2448e4248917a0ea3e3b05bd8/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:14.326454 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf89d57-7ca8-498b-a24a-c07e5216a680 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:14.431724 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf89d57-7ca8-498b-a24a-c07e5216a680 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:14.437629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b856c2-847f-4f68-bbb4-f2619b69dd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:14.450014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b856c2-847f-4f68-bbb4-f2619b69dd95 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:14.450014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b856c2-847f-4f68-bbb4-f2619b69dd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:14.450014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b856c2-847f-4f68-bbb4-f2619b69dd95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:14.450014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b856c2-847f-4f68-bbb4-f2619b69dd95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:14.450014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b856c2-847f-4f68-bbb4-f2619b69dd95 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:14.462625 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 968/1924] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:53:14.470460 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b856c2-847f-4f68-bbb4-f2619b69dd95 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:14.492028 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b856c2-847f-4f68-bbb4-f2619b69dd95 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:14.498461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c42656-d0c4-4531-99f4-b003181057c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:14.499210 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c42656-d0c4-4531-99f4-b003181057c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:14.499551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c42656-d0c4-4531-99f4-b003181057c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:14.499881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c42656-d0c4-4531-99f4-b003181057c7 admin admin] PATH_INFO: `/v3/projects/149ee4cf8d20400c8532289d783ea636/users/d492d8c2448e4248917a0ea3e3b05bd8/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:14.502418 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c42656-d0c4-4531-99f4-b003181057c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=149ee4cf8d20400c8532289d783ea636, user_id=d492d8c2448e4248917a0ea3e3b05bd8, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:14.505490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c42656-d0c4-4531-99f4-b003181057c7 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:14.529745 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 969/1925] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:14 2022] PUT /identity/v3/projects/149ee4cf8d20400c8532289d783ea636/users/d492d8c2448e4248917a0ea3e3b05bd8/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:14.543944 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c48d3-acbe-49ff-8e29-dbaab5cb3a59 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:14.544351 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c48d3-acbe-49ff-8e29-dbaab5cb3a59 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:14.544714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c48d3-acbe-49ff-8e29-dbaab5cb3a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:14.587955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-5b8c48d3-acbe-49ff-8e29-dbaab5cb3a59 None None] MFA Rules not processed for user `d492d8c2448e4248917a0ea3e3b05bd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:14.590042 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-5b8c48d3-acbe-49ff-8e29-dbaab5cb3a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:14.667655 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:14.734603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 957/1926] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:14 2022] PUT /identity/v3/projects/f0d903adbca34fd4b3c1f1e3f838fb7c/users/ff230e7efdcd405e8be7034d567357e1/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:14.744099 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0a4c7-8060-4963-9162-a8b47004b5f6 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:14.744608 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0a4c7-8060-4963-9162-a8b47004b5f6 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:14.745105 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0a4c7-8060-4963-9162-a8b47004b5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:14.756487 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 970/1927] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:14 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:53:14.767491 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c48d3-acbe-49ff-8e29-dbaab5cb3a59 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:14.769553 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-5b8c48d3-acbe-49ff-8e29-dbaab5cb3a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:14.805401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-a1c0a4c7-8060-4963-9162-a8b47004b5f6 None None] MFA Rules not processed for user `ff230e7efdcd405e8be7034d567357e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:14.807135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-a1c0a4c7-8060-4963-9162-a8b47004b5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:14.870237 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c48d3-acbe-49ff-8e29-dbaab5cb3a59 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:14.912934 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 958/1928] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:14.924096 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0a4c7-8060-4963-9162-a8b47004b5f6 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:14.977190 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7222beb2-5787-4a8d-b557-0cfc8e11f505 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:14.978378 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222beb2-5787-4a8d-b557-0cfc8e11f505 service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:14.979004 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222beb2-5787-4a8d-b557-0cfc8e11f505 service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:14.979457 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222beb2-5787-4a8d-b557-0cfc8e11f505 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:15.028675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0a4c7-8060-4963-9162-a8b47004b5f6 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:15.033875 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf04b5-00ae-4cc7-a7f6-c44d826c2d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:15.034519 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf04b5-00ae-4cc7-a7f6-c44d826c2d36 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:15.034840 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf04b5-00ae-4cc7-a7f6-c44d826c2d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:15.035117 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf04b5-00ae-4cc7-a7f6-c44d826c2d36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:15.035759 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf04b5-00ae-4cc7-a7f6-c44d826c2d36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:15.039007 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf04b5-00ae-4cc7-a7f6-c44d826c2d36 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:15.077132 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7222beb2-5787-4a8d-b557-0cfc8e11f505 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:15.079801 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7222beb2-5787-4a8d-b557-0cfc8e11f505 service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:15.098849 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 971/1929] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:14 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:15.167271 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7222beb2-5787-4a8d-b557-0cfc8e11f505 service cinder] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:15.170772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-7222beb2-5787-4a8d-b557-0cfc8e11f505 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:15.177900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 959/1930] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:53:14 2022] POST /identity/v3/projects => generated 375 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:53:15.180892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf04b5-00ae-4cc7-a7f6-c44d826c2d36 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:15.193122 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7222beb2-5787-4a8d-b557-0cfc8e11f505 service cinder] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:15.197700 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:15.305810 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b135cec-4142-4d2f-b353-cb1f14c31b33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:15.306801 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b135cec-4142-4d2f-b353-cb1f14c31b33 service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:15.307347 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b135cec-4142-4d2f-b353-cb1f14c31b33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:15.307922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b135cec-4142-4d2f-b353-cb1f14c31b33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:15.323360 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf04b5-00ae-4cc7-a7f6-c44d826c2d36 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:15.329341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d3db6-6f76-420d-95af-0234fc926162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:15.330008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d3db6-6f76-420d-95af-0234fc926162 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:15.330342 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d3db6-6f76-420d-95af-0234fc926162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:15.330554 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d3db6-6f76-420d-95af-0234fc926162 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:15.331335 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d3db6-6f76-420d-95af-0234fc926162 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:15.333289 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d3db6-6f76-420d-95af-0234fc926162 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:15.404565 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b135cec-4142-4d2f-b353-cb1f14c31b33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:15.406063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b135cec-4142-4d2f-b353-cb1f14c31b33 service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:15.419779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 972/1931] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:15 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:53:15.561491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 960/1932] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:53:15 2022] POST /identity/v3/users => generated 294 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:15.564958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b135cec-4142-4d2f-b353-cb1f14c31b33 service cinder] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:15.578162 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b135cec-4142-4d2f-b353-cb1f14c31b33 service cinder] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:15.584223 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa35ea0-d4b2-484e-b35c-b8aec6632be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:15.585191 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa35ea0-d4b2-484e-b35c-b8aec6632be0 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:15.585581 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa35ea0-d4b2-484e-b35c-b8aec6632be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:15.586019 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa35ea0-d4b2-484e-b35c-b8aec6632be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:15.586749 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa35ea0-d4b2-484e-b35c-b8aec6632be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:15.588635 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa35ea0-d4b2-484e-b35c-b8aec6632be0 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:15.607380 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d3db6-6f76-420d-95af-0234fc926162 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:15.608165 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 973/1933] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:15.648588 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d3db6-6f76-420d-95af-0234fc926162 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:15.670333 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a4c4d-21c5-4dff-98fd-b3cd5d05095d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:15.671450 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a4c4d-21c5-4dff-98fd-b3cd5d05095d admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:15.672081 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a4c4d-21c5-4dff-98fd-b3cd5d05095d admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:15.672242 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a4c4d-21c5-4dff-98fd-b3cd5d05095d admin admin] PATH_INFO: `/v3/projects/0d8552e71a0e4c5fa22aa6f21fb37c29/users/56dc4b6625a941dbb01c8db94bf9fcbf/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:15.707830 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a4c4d-21c5-4dff-98fd-b3cd5d05095d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d8552e71a0e4c5fa22aa6f21fb37c29, user_id=56dc4b6625a941dbb01c8db94bf9fcbf, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:15.720610 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a4c4d-21c5-4dff-98fd-b3cd5d05095d admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:15.836894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa35ea0-d4b2-484e-b35c-b8aec6632be0 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:15.839386 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-3fa35ea0-d4b2-484e-b35c-b8aec6632be0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:15.842142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 961/1934] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:15 2022] PUT /identity/v3/projects/0d8552e71a0e4c5fa22aa6f21fb37c29/users/56dc4b6625a941dbb01c8db94bf9fcbf/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:15.851223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a4c4d-21c5-4dff-98fd-b3cd5d05095d admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:15.861881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa35ea0-d4b2-484e-b35c-b8aec6632be0 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:15.968237 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87955463-08c0-447f-a3ae-cd0ba0b6a29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:15.969854 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87955463-08c0-447f-a3ae-cd0ba0b6a29e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:15.970334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87955463-08c0-447f-a3ae-cd0ba0b6a29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:15.971053 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87955463-08c0-447f-a3ae-cd0ba0b6a29e admin admin] PATH_INFO: `/v3/users/5a1203cf3b194344822af1836c0a1d68` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:15.976239 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87955463-08c0-447f-a3ae-cd0ba0b6a29e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a1203cf3b194344822af1836c0a1d68)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:15.980154 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87955463-08c0-447f-a3ae-cd0ba0b6a29e admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:15.981966 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a4c4d-21c5-4dff-98fd-b3cd5d05095d admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:15.990610 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59d4a4-678f-4691-af1c-fd5b0b234440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:15.991763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59d4a4-678f-4691-af1c-fd5b0b234440 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:15.992490 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59d4a4-678f-4691-af1c-fd5b0b234440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:15.992764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59d4a4-678f-4691-af1c-fd5b0b234440 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:15.993963 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59d4a4-678f-4691-af1c-fd5b0b234440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:15.996994 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59d4a4-678f-4691-af1c-fd5b0b234440 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:16.017352 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 962/1935] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:53:16.022406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59d4a4-678f-4691-af1c-fd5b0b234440 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:16.037448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59d4a4-678f-4691-af1c-fd5b0b234440 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:16.042958 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93b7ee-bb34-41f7-93a4-9ddf43396710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:16.043665 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93b7ee-bb34-41f7-93a4-9ddf43396710 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:16.043973 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93b7ee-bb34-41f7-93a4-9ddf43396710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:16.044434 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93b7ee-bb34-41f7-93a4-9ddf43396710 admin admin] PATH_INFO: `/v3/projects/0d8552e71a0e4c5fa22aa6f21fb37c29/users/56dc4b6625a941dbb01c8db94bf9fcbf/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:16.046639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93b7ee-bb34-41f7-93a4-9ddf43396710 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d8552e71a0e4c5fa22aa6f21fb37c29, user_id=56dc4b6625a941dbb01c8db94bf9fcbf, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:16.049687 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93b7ee-bb34-41f7-93a4-9ddf43396710 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:16.062277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 963/1936] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:16 2022] PUT /identity/v3/projects/0d8552e71a0e4c5fa22aa6f21fb37c29/users/56dc4b6625a941dbb01c8db94bf9fcbf/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:16.074101 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306cfba-b8b9-4416-baeb-0da6eb9f9bbc None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:16.074368 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306cfba-b8b9-4416-baeb-0da6eb9f9bbc None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:16.074690 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306cfba-b8b9-4416-baeb-0da6eb9f9bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:16.111299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-1306cfba-b8b9-4416-baeb-0da6eb9f9bbc None None] MFA Rules not processed for user `56dc4b6625a941dbb01c8db94bf9fcbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:16.113524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-1306cfba-b8b9-4416-baeb-0da6eb9f9bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:16.220375 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 964/1937] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:53:16.236510 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306cfba-b8b9-4416-baeb-0da6eb9f9bbc None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:16.300620 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-87955463-08c0-447f-a3ae-cd0ba0b6a29e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a1203cf3b194344822af1836c0a1d68'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:16.305679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306cfba-b8b9-4416-baeb-0da6eb9f9bbc None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:16.311898 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae927fd0-4007-475a-a24f-783983c02a5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:16.312484 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae927fd0-4007-475a-a24f-783983c02a5f service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:16.312967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae927fd0-4007-475a-a24f-783983c02a5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:16.313248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae927fd0-4007-475a-a24f-783983c02a5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:16.385498 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae927fd0-4007-475a-a24f-783983c02a5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:16.387649 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae927fd0-4007-475a-a24f-783983c02a5f service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:16.402308 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 965/1938] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:53:16.490049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-87955463-08c0-447f-a3ae-cd0ba0b6a29e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a1203cf3b194344822af1836c0a1d68'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:16.491242 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-87955463-08c0-447f-a3ae-cd0ba0b6a29e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a1203cf3b194344822af1836c0a1d68'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:16.503744 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-87955463-08c0-447f-a3ae-cd0ba0b6a29e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a1203cf3b194344822af1836c0a1d68'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:16.524761 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-87955463-08c0-447f-a3ae-cd0ba0b6a29e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a1203cf3b194344822af1836c0a1d68'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:16.536134 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 974/1939] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:53:15 2022] DELETE /identity/v3/users/5a1203cf3b194344822af1836c0a1d68 => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:16.545090 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d890d6-e150-4e04-95f5-f5df81c91b02 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:16.545645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d890d6-e150-4e04-95f5-f5df81c91b02 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:16.546487 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d890d6-e150-4e04-95f5-f5df81c91b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:16.593238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-87d890d6-e150-4e04-95f5-f5df81c91b02 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:16.595390 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-87d890d6-e150-4e04-95f5-f5df81c91b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:16.714157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 966/1940] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:16.739652 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87955463-08c0-447f-a3ae-cd0ba0b6a29e admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:16.744123 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-87955463-08c0-447f-a3ae-cd0ba0b6a29e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:16.846282 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87955463-08c0-447f-a3ae-cd0ba0b6a29e admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:16.851759 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae95e1-3f08-4b8b-bced-8abe4f599545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:16.852563 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae95e1-3f08-4b8b-bced-8abe4f599545 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:16.852936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae95e1-3f08-4b8b-bced-8abe4f599545 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:16.853399 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae95e1-3f08-4b8b-bced-8abe4f599545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:16.869510 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cae95e1-3f08-4b8b-bced-8abe4f599545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:16.873223 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cae95e1-3f08-4b8b-bced-8abe4f599545 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:16.894693 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 975/1941] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:17.126302 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:17.259980 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:17.570858 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d890d6-e150-4e04-95f5-f5df81c91b02 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:17.573248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-87d890d6-e150-4e04-95f5-f5df81c91b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:17.596817 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d890d6-e150-4e04-95f5-f5df81c91b02 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:17.607167 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1ceba-c733-4214-9a0b-2672c5de5a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:17.608311 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1ceba-c733-4214-9a0b-2672c5de5a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:17.608540 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1ceba-c733-4214-9a0b-2672c5de5a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:17.608993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1ceba-c733-4214-9a0b-2672c5de5a80 admin admin] PATH_INFO: `/v3/projects/ff651eab623d4748b16914e5ff4941a1` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:17.610603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec1ceba-c733-4214-9a0b-2672c5de5a80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff651eab623d4748b16914e5ff4941a1)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:17.613064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec1ceba-c733-4214-9a0b-2672c5de5a80 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:17.787423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-aec1ceba-c733-4214-9a0b-2672c5de5a80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff651eab623d4748b16914e5ff4941a1', 'request_id': 'req-aec1ceba-c733-4214-9a0b-2672c5de5a80'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:17.797518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-aec1ceba-c733-4214-9a0b-2672c5de5a80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff651eab623d4748b16914e5ff4941a1', 'request_id': 'req-aec1ceba-c733-4214-9a0b-2672c5de5a80'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:17.809193 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-aec1ceba-c733-4214-9a0b-2672c5de5a80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff651eab623d4748b16914e5ff4941a1', 'request_id': 'req-aec1ceba-c733-4214-9a0b-2672c5de5a80'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:17.905174 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-aec1ceba-c733-4214-9a0b-2672c5de5a80 admin admin] The token cache is being invalidate because project ff651eab623d4748b16914e5ff4941a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:53:17.905591 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-aec1ceba-c733-4214-9a0b-2672c5de5a80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:17.906594 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-aec1ceba-c733-4214-9a0b-2672c5de5a80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:17.908507 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 967/1942] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:53:17 2022] DELETE /identity/v3/projects/ff651eab623d4748b16914e5ff4941a1 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:17.914851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae95e1-3f08-4b8b-bced-8abe4f599545 service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:17.931174 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae95e1-3f08-4b8b-bced-8abe4f599545 service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:18.016473 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac377f9-346f-46db-9dcf-f3cb6802a7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:18.017309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac377f9-346f-46db-9dcf-f3cb6802a7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:18.017786 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac377f9-346f-46db-9dcf-f3cb6802a7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:18.018354 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac377f9-346f-46db-9dcf-f3cb6802a7ca admin admin] PATH_INFO: `/v3/users/612ed8f6a9694d739a975e8e7d8356da` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:18.020380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac377f9-346f-46db-9dcf-f3cb6802a7ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=612ed8f6a9694d739a975e8e7d8356da)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:18.023654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac377f9-346f-46db-9dcf-f3cb6802a7ca admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:18.318106 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-fac377f9-346f-46db-9dcf-f3cb6802a7ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '612ed8f6a9694d739a975e8e7d8356da'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:18.463185 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-fac377f9-346f-46db-9dcf-f3cb6802a7ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '612ed8f6a9694d739a975e8e7d8356da'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:18.463185 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-fac377f9-346f-46db-9dcf-f3cb6802a7ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '612ed8f6a9694d739a975e8e7d8356da'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:18.483085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-fac377f9-346f-46db-9dcf-f3cb6802a7ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '612ed8f6a9694d739a975e8e7d8356da'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:18.518496 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-fac377f9-346f-46db-9dcf-f3cb6802a7ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '612ed8f6a9694d739a975e8e7d8356da'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:18.542762 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 976/1943] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:53:17 2022] DELETE /identity/v3/users/612ed8f6a9694d739a975e8e7d8356da => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:18.858353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:18.972972 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:19.298457 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1ceba-c733-4214-9a0b-2672c5de5a80 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:19.301688 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-aec1ceba-c733-4214-9a0b-2672c5de5a80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:19.332391 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1ceba-c733-4214-9a0b-2672c5de5a80 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:19.384668 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:19.431462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ac4e2-f7b7-410a-be83-c428a7bc893c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:19.432657 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ac4e2-f7b7-410a-be83-c428a7bc893c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:19.433147 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ac4e2-f7b7-410a-be83-c428a7bc893c admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:19.433718 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ac4e2-f7b7-410a-be83-c428a7bc893c admin admin] PATH_INFO: `/v3/projects/b73dd6be895544749e1bc2e901770191` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:19.435894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ac4e2-f7b7-410a-be83-c428a7bc893c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b73dd6be895544749e1bc2e901770191)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:19.440000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ac4e2-f7b7-410a-be83-c428a7bc893c admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:19.621612 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e83ac4e2-f7b7-410a-be83-c428a7bc893c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b73dd6be895544749e1bc2e901770191', 'request_id': 'req-e83ac4e2-f7b7-410a-be83-c428a7bc893c'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:19.652441 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e83ac4e2-f7b7-410a-be83-c428a7bc893c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b73dd6be895544749e1bc2e901770191', 'request_id': 'req-e83ac4e2-f7b7-410a-be83-c428a7bc893c'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:19.659714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e83ac4e2-f7b7-410a-be83-c428a7bc893c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b73dd6be895544749e1bc2e901770191', 'request_id': 'req-e83ac4e2-f7b7-410a-be83-c428a7bc893c'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:19.751945 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e83ac4e2-f7b7-410a-be83-c428a7bc893c admin admin] The token cache is being invalidate because project b73dd6be895544749e1bc2e901770191 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:53:19.752342 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e83ac4e2-f7b7-410a-be83-c428a7bc893c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:19.753208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e83ac4e2-f7b7-410a-be83-c428a7bc893c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:19.755136 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 968/1944] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:53:19 2022] DELETE /identity/v3/projects/b73dd6be895544749e1bc2e901770191 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:19.760908 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5437ee-b0b1-46ee-9630-16a7379c42c2 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:19.761358 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5437ee-b0b1-46ee-9630-16a7379c42c2 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:19.761708 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5437ee-b0b1-46ee-9630-16a7379c42c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:19.765833 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:19.786274 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-6f5437ee-b0b1-46ee-9630-16a7379c42c2 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:19.788279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-6f5437ee-b0b1-46ee-9630-16a7379c42c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:19.869783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 977/1945] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:53:19.881599 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d9858-448c-4639-9d5f-74f173c7b7ee None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:19.881599 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d9858-448c-4639-9d5f-74f173c7b7ee None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:19.882645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d9858-448c-4639-9d5f-74f173c7b7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:19.913975 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-a81d9858-448c-4639-9d5f-74f173c7b7ee None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:19.916196 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-a81d9858-448c-4639-9d5f-74f173c7b7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:19.943910 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 969/1946] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:53:19.950008 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5437ee-b0b1-46ee-9630-16a7379c42c2 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:19.968784 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5437ee-b0b1-46ee-9630-16a7379c42c2 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:19.975611 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6e4c0-7e74-4f84-9268-4a65654d0b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:19.978126 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6e4c0-7e74-4f84-9268-4a65654d0b4c admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:19.978585 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6e4c0-7e74-4f84-9268-4a65654d0b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:19.980108 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6e4c0-7e74-4f84-9268-4a65654d0b4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:19.981162 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef6e4c0-7e74-4f84-9268-4a65654d0b4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:53:19.981630 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef6e4c0-7e74-4f84-9268-4a65654d0b4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:19.984836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef6e4c0-7e74-4f84-9268-4a65654d0b4c admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:19.997338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d9858-448c-4639-9d5f-74f173c7b7ee None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:19.999951 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-a81d9858-448c-4639-9d5f-74f173c7b7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:20.001615 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 978/1947] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:53:20.009513 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6e4c0-7e74-4f84-9268-4a65654d0b4c admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:20.023036 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6e4c0-7e74-4f84-9268-4a65654d0b4c admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:20.030162 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d9858-448c-4639-9d5f-74f173c7b7ee None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:20.037120 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e66bac-bf3a-4ad4-a691-09bdfff923eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:20.038216 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e66bac-bf3a-4ad4-a691-09bdfff923eb admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:20.038732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e66bac-bf3a-4ad4-a691-09bdfff923eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:20.039182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e66bac-bf3a-4ad4-a691-09bdfff923eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:20.039990 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530d2b9-4bf0-4b67-be4e-2df9f8280698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:20.043050 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530d2b9-4bf0-4b67-be4e-2df9f8280698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:20.043050 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530d2b9-4bf0-4b67-be4e-2df9f8280698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:20.043050 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530d2b9-4bf0-4b67-be4e-2df9f8280698 admin admin] PATH_INFO: `/v3/users/ff230e7efdcd405e8be7034d567357e1` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:20.043482 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e66bac-bf3a-4ad4-a691-09bdfff923eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:20.044700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530d2b9-4bf0-4b67-be4e-2df9f8280698 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff230e7efdcd405e8be7034d567357e1)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:20.045583 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e66bac-bf3a-4ad4-a691-09bdfff923eb admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:20.049277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530d2b9-4bf0-4b67-be4e-2df9f8280698 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:20.174972 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 979/1948] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:53:20 2022] POST /identity/v3/projects => generated 369 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:53:20.181395 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e66bac-bf3a-4ad4-a691-09bdfff923eb admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:20.281753 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e66bac-bf3a-4ad4-a691-09bdfff923eb admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:20.288155 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7530d2b9-4bf0-4b67-be4e-2df9f8280698 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff230e7efdcd405e8be7034d567357e1'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:20.288707 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517158d1-60bc-4897-bb89-bd268fe47513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:20.289421 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517158d1-60bc-4897-bb89-bd268fe47513 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:20.289989 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517158d1-60bc-4897-bb89-bd268fe47513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:20.290238 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517158d1-60bc-4897-bb89-bd268fe47513 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:20.291145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517158d1-60bc-4897-bb89-bd268fe47513 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:20.293666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517158d1-60bc-4897-bb89-bd268fe47513 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:20.469500 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7530d2b9-4bf0-4b67-be4e-2df9f8280698 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff230e7efdcd405e8be7034d567357e1'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:20.470499 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7530d2b9-4bf0-4b67-be4e-2df9f8280698 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff230e7efdcd405e8be7034d567357e1'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:20.477466 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 980/1949] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:53:20 2022] POST /identity/v3/users => generated 285 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:20.482175 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517158d1-60bc-4897-bb89-bd268fe47513 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:20.482420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7530d2b9-4bf0-4b67-be4e-2df9f8280698 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff230e7efdcd405e8be7034d567357e1'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:20.483650 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7530d2b9-4bf0-4b67-be4e-2df9f8280698 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff230e7efdcd405e8be7034d567357e1'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:20.484012 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-517158d1-60bc-4897-bb89-bd268fe47513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:20.498673 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517158d1-60bc-4897-bb89-bd268fe47513 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:20.504730 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f32f37-74b5-4b44-859f-773d27f4f971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:20.506124 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f32f37-74b5-4b44-859f-773d27f4f971 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:20.506673 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f32f37-74b5-4b44-859f-773d27f4f971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:20.506811 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f32f37-74b5-4b44-859f-773d27f4f971 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:20.507538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f32f37-74b5-4b44-859f-773d27f4f971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:20.509536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f32f37-74b5-4b44-859f-773d27f4f971 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:20.518314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 970/1950] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:53:19 2022] DELETE /identity/v3/users/ff230e7efdcd405e8be7034d567357e1 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:20.525209 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b726d9-c5de-4194-8cef-057247db36bc None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:20.525470 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b726d9-c5de-4194-8cef-057247db36bc None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:20.525777 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b726d9-c5de-4194-8cef-057247db36bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:20.533405 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 981/1951] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:53:20.535521 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f32f37-74b5-4b44-859f-773d27f4f971 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:20.551521 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f32f37-74b5-4b44-859f-773d27f4f971 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:20.552991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-94b726d9-c5de-4194-8cef-057247db36bc None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:20.554381 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-94b726d9-c5de-4194-8cef-057247db36bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:20.638334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affeec99-4f03-49ae-906f-c9d57a89d00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:20.639436 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affeec99-4f03-49ae-906f-c9d57a89d00e admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:20.639879 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affeec99-4f03-49ae-906f-c9d57a89d00e admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:20.640323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affeec99-4f03-49ae-906f-c9d57a89d00e admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:20.641317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affeec99-4f03-49ae-906f-c9d57a89d00e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:20.642705 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 971/1952] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:53:20.644094 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affeec99-4f03-49ae-906f-c9d57a89d00e admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:20.657908 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b726d9-c5de-4194-8cef-057247db36bc None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:20.659517 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-94b726d9-c5de-4194-8cef-057247db36bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:20.667447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 982/1953] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:20.671500 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affeec99-4f03-49ae-906f-c9d57a89d00e admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:20.672281 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:20.687858 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affeec99-4f03-49ae-906f-c9d57a89d00e admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:20.707298 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0373b23-dbb7-4992-93e9-4c9fb370e873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:20.708844 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0373b23-dbb7-4992-93e9-4c9fb370e873 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:20.709846 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0373b23-dbb7-4992-93e9-4c9fb370e873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:20.710533 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0373b23-dbb7-4992-93e9-4c9fb370e873 admin admin] PATH_INFO: `/v3/projects/90e6b59b14594b7c9ba05790eb5fda3c/users/38cf55e5e4a9466ab95b839f8208abee/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:20.746622 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0373b23-dbb7-4992-93e9-4c9fb370e873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90e6b59b14594b7c9ba05790eb5fda3c, user_id=38cf55e5e4a9466ab95b839f8208abee, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:20.754909 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0373b23-dbb7-4992-93e9-4c9fb370e873 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:20.799740 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b726d9-c5de-4194-8cef-057247db36bc None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:20.806779 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0032ed-c5a4-4001-bf92-c0fe89e3da87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:20.807989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0032ed-c5a4-4001-bf92-c0fe89e3da87 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:20.808539 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0032ed-c5a4-4001-bf92-c0fe89e3da87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:20.809089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0032ed-c5a4-4001-bf92-c0fe89e3da87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:20.826085 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0032ed-c5a4-4001-bf92-c0fe89e3da87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:20.827872 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0032ed-c5a4-4001-bf92-c0fe89e3da87 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:20.854194 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 972/1954] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:53:20.861652 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 983/1955] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:20 2022] PUT /identity/v3/projects/90e6b59b14594b7c9ba05790eb5fda3c/users/38cf55e5e4a9466ab95b839f8208abee/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:20.878603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afe617-a433-4f4f-a9ae-2b010ef85aa5 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:20.878816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:20.878900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afe617-a433-4f4f-a9ae-2b010ef85aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:20.881468 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afe617-a433-4f4f-a9ae-2b010ef85aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:20.932661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-26afe617-a433-4f4f-a9ae-2b010ef85aa5 None None] MFA Rules not processed for user `38cf55e5e4a9466ab95b839f8208abee`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:20.935119 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-26afe617-a433-4f4f-a9ae-2b010ef85aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:21.049327 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 973/1956] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:21.056341 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0373b23-dbb7-4992-93e9-4c9fb370e873 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:21.213275 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0373b23-dbb7-4992-93e9-4c9fb370e873 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:21.223333 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bbfa5-dc9e-49e4-af4e-da3f4a625dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:21.224288 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bbfa5-dc9e-49e4-af4e-da3f4a625dad admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:21.224762 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bbfa5-dc9e-49e4-af4e-da3f4a625dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:21.225239 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bbfa5-dc9e-49e4-af4e-da3f4a625dad admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:21.226427 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699bbfa5-dc9e-49e4-af4e-da3f4a625dad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:21.230326 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699bbfa5-dc9e-49e4-af4e-da3f4a625dad admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:21.404019 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 984/1957] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:53:21 2022] POST /identity/v3/projects => generated 367 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:21.409902 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afe617-a433-4f4f-a9ae-2b010ef85aa5 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:21.429158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afe617-a433-4f4f-a9ae-2b010ef85aa5 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:21.502968 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339f996-1f6c-4eba-80ac-bbf59d683cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:21.504203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339f996-1f6c-4eba-80ac-bbf59d683cee admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:21.504623 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339f996-1f6c-4eba-80ac-bbf59d683cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:21.506312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339f996-1f6c-4eba-80ac-bbf59d683cee admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:21.507320 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339f996-1f6c-4eba-80ac-bbf59d683cee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:21.509330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339f996-1f6c-4eba-80ac-bbf59d683cee admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:21.654290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 974/1958] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:53:21 2022] POST /identity/v3/users => generated 290 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:53:21.659457 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bbfa5-dc9e-49e4-af4e-da3f4a625dad admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:21.680521 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bbfa5-dc9e-49e4-af4e-da3f4a625dad admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:21.689614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab97ec-7af9-498b-82cd-ec9bb4e18f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:21.690786 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab97ec-7af9-498b-82cd-ec9bb4e18f89 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:21.691284 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab97ec-7af9-498b-82cd-ec9bb4e18f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:21.691780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab97ec-7af9-498b-82cd-ec9bb4e18f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:21.693447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab97ec-7af9-498b-82cd-ec9bb4e18f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:21.696723 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab97ec-7af9-498b-82cd-ec9bb4e18f89 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:21.710095 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339f996-1f6c-4eba-80ac-bbf59d683cee admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:21.713353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-3339f996-1f6c-4eba-80ac-bbf59d683cee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:21.726963 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 985/1959] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:21.733939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab97ec-7af9-498b-82cd-ec9bb4e18f89 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:21.735427 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339f996-1f6c-4eba-80ac-bbf59d683cee admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:21.744369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad3f33-0aab-4b6f-a73f-25f3bd8bef08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:21.745493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad3f33-0aab-4b6f-a73f-25f3bd8bef08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:21.745978 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad3f33-0aab-4b6f-a73f-25f3bd8bef08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:21.751882 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad3f33-0aab-4b6f-a73f-25f3bd8bef08 admin admin] PATH_INFO: `/v3/projects/f0d903adbca34fd4b3c1f1e3f838fb7c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:21.751882 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad3f33-0aab-4b6f-a73f-25f3bd8bef08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0d903adbca34fd4b3c1f1e3f838fb7c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:21.754789 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad3f33-0aab-4b6f-a73f-25f3bd8bef08 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:21.762203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab97ec-7af9-498b-82cd-ec9bb4e18f89 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:21.773598 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f5aaf-41c6-40a1-b272-fda10fd78d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:21.775750 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f5aaf-41c6-40a1-b272-fda10fd78d21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:21.776593 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f5aaf-41c6-40a1-b272-fda10fd78d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:21.777219 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f5aaf-41c6-40a1-b272-fda10fd78d21 admin admin] PATH_INFO: `/v3/projects/bbebc6be7c4f4837aba22e1304dd2b9c/users/c272f90d4baa4f22990ced81273e8484/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:21.814649 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f5aaf-41c6-40a1-b272-fda10fd78d21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbebc6be7c4f4837aba22e1304dd2b9c, user_id=c272f90d4baa4f22990ced81273e8484, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:21.822504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f5aaf-41c6-40a1-b272-fda10fd78d21 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:21.968694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 986/1960] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:21 2022] PUT /identity/v3/projects/bbebc6be7c4f4837aba22e1304dd2b9c/users/c272f90d4baa4f22990ced81273e8484/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:21.970006 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f5aaf-41c6-40a1-b272-fda10fd78d21 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:21.973378 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-112f5aaf-41c6-40a1-b272-fda10fd78d21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:21.985293 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-28ad3f33-0aab-4b6f-a73f-25f3bd8bef08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0d903adbca34fd4b3c1f1e3f838fb7c', 'request_id': 'req-28ad3f33-0aab-4b6f-a73f-25f3bd8bef08'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:21.996021 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-28ad3f33-0aab-4b6f-a73f-25f3bd8bef08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0d903adbca34fd4b3c1f1e3f838fb7c', 'request_id': 'req-28ad3f33-0aab-4b6f-a73f-25f3bd8bef08'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:22.008516 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-28ad3f33-0aab-4b6f-a73f-25f3bd8bef08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0d903adbca34fd4b3c1f1e3f838fb7c', 'request_id': 'req-28ad3f33-0aab-4b6f-a73f-25f3bd8bef08'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:22.065159 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:22.098791 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-28ad3f33-0aab-4b6f-a73f-25f3bd8bef08 admin admin] The token cache is being invalidate because project f0d903adbca34fd4b3c1f1e3f838fb7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:53:22.099007 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-28ad3f33-0aab-4b6f-a73f-25f3bd8bef08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:22.099900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-28ad3f33-0aab-4b6f-a73f-25f3bd8bef08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:22.103233 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f5aaf-41c6-40a1-b272-fda10fd78d21 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:22.108755 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 975/1961] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:53:21 2022] DELETE /identity/v3/projects/f0d903adbca34fd4b3c1f1e3f838fb7c => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:22.110311 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad3f33-0aab-4b6f-a73f-25f3bd8bef08 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:22.112673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-28ad3f33-0aab-4b6f-a73f-25f3bd8bef08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:22.219868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:22.240151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1627a29-1994-4012-8bd1-84fc5c6cb845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:22.241837 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1627a29-1994-4012-8bd1-84fc5c6cb845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:22.242843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1627a29-1994-4012-8bd1-84fc5c6cb845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:22.243560 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1627a29-1994-4012-8bd1-84fc5c6cb845 admin admin] PATH_INFO: `/v3/users/8effb799317c4e71bd68c977e30b4d29` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:22.246487 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1627a29-1994-4012-8bd1-84fc5c6cb845 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8effb799317c4e71bd68c977e30b4d29)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:22.251245 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1627a29-1994-4012-8bd1-84fc5c6cb845 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:22.253553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad3f33-0aab-4b6f-a73f-25f3bd8bef08 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:22.262600 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f16d80-769f-448e-bbb9-e22e0cc81967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:22.263841 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f16d80-769f-448e-bbb9-e22e0cc81967 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:22.264602 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f16d80-769f-448e-bbb9-e22e0cc81967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:22.265139 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f16d80-769f-448e-bbb9-e22e0cc81967 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:22.266834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f16d80-769f-448e-bbb9-e22e0cc81967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:22.269026 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f16d80-769f-448e-bbb9-e22e0cc81967 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:22.285341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 976/1962] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:22.286015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f16d80-769f-448e-bbb9-e22e0cc81967 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:22.288702 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d1f16d80-769f-448e-bbb9-e22e0cc81967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:22.310890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f16d80-769f-448e-bbb9-e22e0cc81967 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:22.318737 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52647fd-3c6f-4f41-a549-1d705a04c525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:22.319711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52647fd-3c6f-4f41-a549-1d705a04c525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:22.320220 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52647fd-3c6f-4f41-a549-1d705a04c525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:22.320550 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52647fd-3c6f-4f41-a549-1d705a04c525 admin admin] PATH_INFO: `/v3/users/56dc4b6625a941dbb01c8db94bf9fcbf` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:22.322011 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52647fd-3c6f-4f41-a549-1d705a04c525 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56dc4b6625a941dbb01c8db94bf9fcbf)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:22.324218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52647fd-3c6f-4f41-a549-1d705a04c525 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:22.508421 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:22.515613 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b1627a29-1994-4012-8bd1-84fc5c6cb845 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8effb799317c4e71bd68c977e30b4d29'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:22.587774 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e52647fd-3c6f-4f41-a549-1d705a04c525 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56dc4b6625a941dbb01c8db94bf9fcbf'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:22.603115 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b1627a29-1994-4012-8bd1-84fc5c6cb845 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8effb799317c4e71bd68c977e30b4d29'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:22.604781 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b1627a29-1994-4012-8bd1-84fc5c6cb845 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8effb799317c4e71bd68c977e30b4d29'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:22.606639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:22.617088 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b1627a29-1994-4012-8bd1-84fc5c6cb845 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8effb799317c4e71bd68c977e30b4d29'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:22.637995 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b1627a29-1994-4012-8bd1-84fc5c6cb845 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8effb799317c4e71bd68c977e30b4d29'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:22.646893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 987/1963] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:53:21 2022] DELETE /identity/v3/users/8effb799317c4e71bd68c977e30b4d29 => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:22.647776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1627a29-1994-4012-8bd1-84fc5c6cb845 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:22.649266 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b1627a29-1994-4012-8bd1-84fc5c6cb845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:22.756541 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1627a29-1994-4012-8bd1-84fc5c6cb845 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:22.764884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57edf0d-d457-41bc-b5e6-3546a7ec4f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:22.766095 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57edf0d-d457-41bc-b5e6-3546a7ec4f31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:22.766551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57edf0d-d457-41bc-b5e6-3546a7ec4f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:22.767116 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57edf0d-d457-41bc-b5e6-3546a7ec4f31 admin admin] PATH_INFO: `/v3/projects/bbebc6be7c4f4837aba22e1304dd2b9c/users/c272f90d4baa4f22990ced81273e8484/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:22.769731 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57edf0d-d457-41bc-b5e6-3546a7ec4f31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbebc6be7c4f4837aba22e1304dd2b9c, user_id=c272f90d4baa4f22990ced81273e8484, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:22.772576 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e52647fd-3c6f-4f41-a549-1d705a04c525 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56dc4b6625a941dbb01c8db94bf9fcbf'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:22.773126 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57edf0d-d457-41bc-b5e6-3546a7ec4f31 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:22.773765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e52647fd-3c6f-4f41-a549-1d705a04c525 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56dc4b6625a941dbb01c8db94bf9fcbf'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:22.789159 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 988/1964] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:22 2022] PUT /identity/v3/projects/bbebc6be7c4f4837aba22e1304dd2b9c/users/c272f90d4baa4f22990ced81273e8484/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:22.790398 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91831eb2-92e0-4b37-bc43-82dc93ed2c13 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:22.790747 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91831eb2-92e0-4b37-bc43-82dc93ed2c13 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:22.791036 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91831eb2-92e0-4b37-bc43-82dc93ed2c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:22.794798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e52647fd-3c6f-4f41-a549-1d705a04c525 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56dc4b6625a941dbb01c8db94bf9fcbf'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:22.800312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e52647fd-3c6f-4f41-a549-1d705a04c525 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56dc4b6625a941dbb01c8db94bf9fcbf'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:22.831923 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 977/1965] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:53:22 2022] DELETE /identity/v3/users/56dc4b6625a941dbb01c8db94bf9fcbf => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:22.833178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585bd44-dc07-42d6-b12c-11c4a3955408 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:22.833983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585bd44-dc07-42d6-b12c-11c4a3955408 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:22.834253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585bd44-dc07-42d6-b12c-11c4a3955408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:22.836161 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-91831eb2-92e0-4b37-bc43-82dc93ed2c13 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:22.841188 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-91831eb2-92e0-4b37-bc43-82dc93ed2c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:22.874544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-0585bd44-dc07-42d6-b12c-11c4a3955408 None None] MFA Rules not processed for user `c272f90d4baa4f22990ced81273e8484`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:22.876078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-0585bd44-dc07-42d6-b12c-11c4a3955408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:22.951194 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 989/1966] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:22.976116 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91831eb2-92e0-4b37-bc43-82dc93ed2c13 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:22.979910 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-91831eb2-92e0-4b37-bc43-82dc93ed2c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:23.011513 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 978/1967] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:23.030559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585bd44-dc07-42d6-b12c-11c4a3955408 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:23.033705 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-0585bd44-dc07-42d6-b12c-11c4a3955408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:23.090805 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91831eb2-92e0-4b37-bc43-82dc93ed2c13 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:23.098367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c044f73-1252-466a-8ec9-5752d80a3a34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:23.100804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c044f73-1252-466a-8ec9-5752d80a3a34 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:23.101285 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c044f73-1252-466a-8ec9-5752d80a3a34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:23.101727 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c044f73-1252-466a-8ec9-5752d80a3a34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:23.123232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c044f73-1252-466a-8ec9-5752d80a3a34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:23.125830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:23.130719 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c044f73-1252-466a-8ec9-5752d80a3a34 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:23.142213 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585bd44-dc07-42d6-b12c-11c4a3955408 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:23.150276 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9ec23-fa4e-4f2e-94e7-31c1ae5bc92b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:23.151500 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 990/1968] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:23.151721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9ec23-fa4e-4f2e-94e7-31c1ae5bc92b service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:23.152260 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9ec23-fa4e-4f2e-94e7-31c1ae5bc92b service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:23.152735 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9ec23-fa4e-4f2e-94e7-31c1ae5bc92b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:23.169773 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9ec23-fa4e-4f2e-94e7-31c1ae5bc92b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:23.173792 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9ec23-fa4e-4f2e-94e7-31c1ae5bc92b service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:23.195640 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 979/1969] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:23.206290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:23.357265 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c044f73-1252-466a-8ec9-5752d80a3a34 service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:23.360411 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-4c044f73-1252-466a-8ec9-5752d80a3a34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:23.412586 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c044f73-1252-466a-8ec9-5752d80a3a34 service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:23.417975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c67bc-4608-4aeb-a277-9564f17966b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:23.418997 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c67bc-4608-4aeb-a277-9564f17966b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:23.419442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c67bc-4608-4aeb-a277-9564f17966b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:23.419780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c67bc-4608-4aeb-a277-9564f17966b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:23.422057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-473c67bc-4608-4aeb-a277-9564f17966b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:23.526620 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c67bc-4608-4aeb-a277-9564f17966b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:23.528654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c67bc-4608-4aeb-a277-9564f17966b8 service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:23.543920 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 991/1970] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:53:23.879085 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9ec23-fa4e-4f2e-94e7-31c1ae5bc92b service cinder] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:23.882946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-47d9ec23-fa4e-4f2e-94e7-31c1ae5bc92b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:23.911034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9ec23-fa4e-4f2e-94e7-31c1ae5bc92b service cinder] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:23.938245 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5c29a-78eb-47de-ba7b-648c664bd96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:23.939495 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5c29a-78eb-47de-ba7b-648c664bd96e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:23.964779 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5c29a-78eb-47de-ba7b-648c664bd96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:23.964779 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5c29a-78eb-47de-ba7b-648c664bd96e admin admin] PATH_INFO: `/v3/projects/0d8552e71a0e4c5fa22aa6f21fb37c29` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:23.964779 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5c29a-78eb-47de-ba7b-648c664bd96e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d8552e71a0e4c5fa22aa6f21fb37c29)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:23.964779 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5c29a-78eb-47de-ba7b-648c664bd96e admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:24.156994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c67bc-4608-4aeb-a277-9564f17966b8 service cinder] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:24.160719 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-473c67bc-4608-4aeb-a277-9564f17966b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:24.180251 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-06e5c29a-78eb-47de-ba7b-648c664bd96e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d8552e71a0e4c5fa22aa6f21fb37c29', 'request_id': 'req-06e5c29a-78eb-47de-ba7b-648c664bd96e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:24.188966 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-06e5c29a-78eb-47de-ba7b-648c664bd96e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d8552e71a0e4c5fa22aa6f21fb37c29', 'request_id': 'req-06e5c29a-78eb-47de-ba7b-648c664bd96e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:24.190836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c67bc-4608-4aeb-a277-9564f17966b8 service cinder] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:24.201172 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-06e5c29a-78eb-47de-ba7b-648c664bd96e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d8552e71a0e4c5fa22aa6f21fb37c29', 'request_id': 'req-06e5c29a-78eb-47de-ba7b-648c664bd96e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:24.279098 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8e79b-9b3a-44b0-9fca-e1e99c9a501d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:24.280149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8e79b-9b3a-44b0-9fca-e1e99c9a501d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:24.280609 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8e79b-9b3a-44b0-9fca-e1e99c9a501d admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:24.281163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8e79b-9b3a-44b0-9fca-e1e99c9a501d admin admin] PATH_INFO: `/v3/projects/605580fa86e84d289ae3f88a6f4a3ded` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:24.283182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8e79b-9b3a-44b0-9fca-e1e99c9a501d admin admin] RBAC: Authorizing `identity:delete_project(project_id=605580fa86e84d289ae3f88a6f4a3ded)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:24.287614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8e79b-9b3a-44b0-9fca-e1e99c9a501d admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:24.298423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-06e5c29a-78eb-47de-ba7b-648c664bd96e admin admin] The token cache is being invalidate because project 0d8552e71a0e4c5fa22aa6f21fb37c29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:53:24.304739 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-06e5c29a-78eb-47de-ba7b-648c664bd96e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:24.304739 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-06e5c29a-78eb-47de-ba7b-648c664bd96e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:24.304739 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 980/1971] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:53:23 2022] DELETE /identity/v3/projects/0d8552e71a0e4c5fa22aa6f21fb37c29 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:24.307489 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b977a7e-e30d-4690-baac-6572521ec68d None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:24.308558 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b977a7e-e30d-4690-baac-6572521ec68d None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:24.308974 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b977a7e-e30d-4690-baac-6572521ec68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:24.339143 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-9b977a7e-e30d-4690-baac-6572521ec68d None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:24.340342 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-9b977a7e-e30d-4690-baac-6572521ec68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:24.348844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 981/1972] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:24.357893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b7855-8d72-437a-899a-ebd538e2a714 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:24.358323 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b7855-8d72-437a-899a-ebd538e2a714 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:24.358703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b7855-8d72-437a-899a-ebd538e2a714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:24.392951 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-478b7855-8d72-437a-899a-ebd538e2a714 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:24.395454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-478b7855-8d72-437a-899a-ebd538e2a714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:24.419276 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 982/1973] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:53:24.428361 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b7855-8d72-437a-899a-ebd538e2a714 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:24.439968 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b7855-8d72-437a-899a-ebd538e2a714 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:24.447357 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a891196-c55b-4d45-adf5-c168a1104fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:24.449142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a891196-c55b-4d45-adf5-c168a1104fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:24.449142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a891196-c55b-4d45-adf5-c168a1104fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:24.449142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a891196-c55b-4d45-adf5-c168a1104fd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:24.450989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a891196-c55b-4d45-adf5-c168a1104fd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:53:24.451667 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a891196-c55b-4d45-adf5-c168a1104fd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:24.455021 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a891196-c55b-4d45-adf5-c168a1104fd0 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:24.477102 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 983/1974] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:53:24.485419 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a891196-c55b-4d45-adf5-c168a1104fd0 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:24.497247 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a891196-c55b-4d45-adf5-c168a1104fd0 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:24.538851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5fb8e79b-9b3a-44b0-9fca-e1e99c9a501d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '605580fa86e84d289ae3f88a6f4a3ded', 'request_id': 'req-5fb8e79b-9b3a-44b0-9fca-e1e99c9a501d'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:24.581079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e465f-f392-48db-bbba-08a59bd70e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:24.582095 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e465f-f392-48db-bbba-08a59bd70e06 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:24.582406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e465f-f392-48db-bbba-08a59bd70e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:24.582725 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e465f-f392-48db-bbba-08a59bd70e06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:24.583940 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e465f-f392-48db-bbba-08a59bd70e06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:24.586303 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e465f-f392-48db-bbba-08a59bd70e06 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:24.628806 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5fb8e79b-9b3a-44b0-9fca-e1e99c9a501d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '605580fa86e84d289ae3f88a6f4a3ded', 'request_id': 'req-5fb8e79b-9b3a-44b0-9fca-e1e99c9a501d'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:24.638281 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5fb8e79b-9b3a-44b0-9fca-e1e99c9a501d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '605580fa86e84d289ae3f88a6f4a3ded', 'request_id': 'req-5fb8e79b-9b3a-44b0-9fca-e1e99c9a501d'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:24.652047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5fb8e79b-9b3a-44b0-9fca-e1e99c9a501d admin admin] The token cache is being invalidate because project 605580fa86e84d289ae3f88a6f4a3ded was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:53:24.652224 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5fb8e79b-9b3a-44b0-9fca-e1e99c9a501d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:24.652820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5fb8e79b-9b3a-44b0-9fca-e1e99c9a501d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:24.654533 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 992/1975] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:53:24 2022] DELETE /identity/v3/projects/605580fa86e84d289ae3f88a6f4a3ded => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:24.658984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8e79b-9b3a-44b0-9fca-e1e99c9a501d admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:24.660519 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-5fb8e79b-9b3a-44b0-9fca-e1e99c9a501d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:24.675959 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8e79b-9b3a-44b0-9fca-e1e99c9a501d admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:24.682060 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bba381-ecab-4ce2-890b-9456bcc5d265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:24.682810 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bba381-ecab-4ce2-890b-9456bcc5d265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:24.683063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bba381-ecab-4ce2-890b-9456bcc5d265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:24.683336 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bba381-ecab-4ce2-890b-9456bcc5d265 admin admin] PATH_INFO: `/v3/users/d492d8c2448e4248917a0ea3e3b05bd8` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:24.684621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bba381-ecab-4ce2-890b-9456bcc5d265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d492d8c2448e4248917a0ea3e3b05bd8)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:24.686615 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bba381-ecab-4ce2-890b-9456bcc5d265 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:24.761309 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 984/1976] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:53:24 2022] POST /identity/v3/projects => generated 377 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:24.766378 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e465f-f392-48db-bbba-08a59bd70e06 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:24.769386 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-de3e465f-f392-48db-bbba-08a59bd70e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:24.802374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:24.883229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e465f-f392-48db-bbba-08a59bd70e06 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:24.890736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b96bc-424d-464b-9f1a-1e3fcc2e744f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:24.891733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b96bc-424d-464b-9f1a-1e3fcc2e744f admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:24.892298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b96bc-424d-464b-9f1a-1e3fcc2e744f admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:24.892640 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b96bc-424d-464b-9f1a-1e3fcc2e744f admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:24.893752 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b96bc-424d-464b-9f1a-1e3fcc2e744f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:24.897661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b96bc-424d-464b-9f1a-1e3fcc2e744f admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:24.976091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-66bba381-ecab-4ce2-890b-9456bcc5d265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd492d8c2448e4248917a0ea3e3b05bd8'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:25.035592 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 985/1977] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:53:24 2022] POST /identity/v3/users => generated 289 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:25.041373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b96bc-424d-464b-9f1a-1e3fcc2e744f admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:25.051712 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-66bba381-ecab-4ce2-890b-9456bcc5d265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd492d8c2448e4248917a0ea3e3b05bd8'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:25.054268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-66bba381-ecab-4ce2-890b-9456bcc5d265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd492d8c2448e4248917a0ea3e3b05bd8'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:25.059827 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b96bc-424d-464b-9f1a-1e3fcc2e744f admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:25.063297 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-66bba381-ecab-4ce2-890b-9456bcc5d265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd492d8c2448e4248917a0ea3e3b05bd8'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:25.069017 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8e75b-059f-4dd5-bf85-555920368b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:25.070056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8e75b-059f-4dd5-bf85-555920368b8a admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:25.070542 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8e75b-059f-4dd5-bf85-555920368b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:25.071034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8e75b-059f-4dd5-bf85-555920368b8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:25.072139 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8e75b-059f-4dd5-bf85-555920368b8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:25.075134 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8e75b-059f-4dd5-bf85-555920368b8a admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:25.078256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-66bba381-ecab-4ce2-890b-9456bcc5d265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd492d8c2448e4248917a0ea3e3b05bd8'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:25.084905 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 993/1978] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:53:24 2022] DELETE /identity/v3/users/d492d8c2448e4248917a0ea3e3b05bd8 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:25.095850 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 986/1979] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:25.101383 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bba381-ecab-4ce2-890b-9456bcc5d265 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:25.103093 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-66bba381-ecab-4ce2-890b-9456bcc5d265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:25.122532 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:25.206958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bba381-ecab-4ce2-890b-9456bcc5d265 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:25.214504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5da8c-7f0b-441d-8518-63ee1dcf7c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:25.215374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5da8c-7f0b-441d-8518-63ee1dcf7c1b admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:25.215846 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5da8c-7f0b-441d-8518-63ee1dcf7c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:25.216286 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5da8c-7f0b-441d-8518-63ee1dcf7c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:25.217131 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b5da8c-7f0b-441d-8518-63ee1dcf7c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:25.219041 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b5da8c-7f0b-441d-8518-63ee1dcf7c1b admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:25.230789 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 994/1980] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:25.235585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8e75b-059f-4dd5-bf85-555920368b8a admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:25.251499 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8e75b-059f-4dd5-bf85-555920368b8a admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:25.261058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45dea0-30ac-4911-a701-6b7678429f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:25.262193 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45dea0-30ac-4911-a701-6b7678429f4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:25.262748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45dea0-30ac-4911-a701-6b7678429f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:25.263246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45dea0-30ac-4911-a701-6b7678429f4c admin admin] PATH_INFO: `/v3/projects/f05babb5bf2d4497b245ebb198673e88/users/f950627ca8544c2f80f2a4f12a4c0bdb/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:25.291701 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45dea0-30ac-4911-a701-6b7678429f4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f05babb5bf2d4497b245ebb198673e88, user_id=f950627ca8544c2f80f2a4f12a4c0bdb, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:25.294396 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45dea0-30ac-4911-a701-6b7678429f4c admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:25.392876 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 987/1981] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:25 2022] PUT /identity/v3/projects/f05babb5bf2d4497b245ebb198673e88/users/f950627ca8544c2f80f2a4f12a4c0bdb/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:25.421165 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af32544-d012-46c4-86dc-c3b6c66dd8cc None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:25.422008 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af32544-d012-46c4-86dc-c3b6c66dd8cc None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:25.422477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af32544-d012-46c4-86dc-c3b6c66dd8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:25.468707 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-2af32544-d012-46c4-86dc-c3b6c66dd8cc None None] MFA Rules not processed for user `f950627ca8544c2f80f2a4f12a4c0bdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:25.470425 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-2af32544-d012-46c4-86dc-c3b6c66dd8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:25.558480 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 995/1982] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:25.582400 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45dea0-30ac-4911-a701-6b7678429f4c admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:25.584146 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-3f45dea0-30ac-4911-a701-6b7678429f4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:25.671430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45dea0-30ac-4911-a701-6b7678429f4c admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:25.682065 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d56c80-d834-41b2-8694-2267f7faf9c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:25.682065 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d56c80-d834-41b2-8694-2267f7faf9c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:25.682065 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d56c80-d834-41b2-8694-2267f7faf9c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:25.682065 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d56c80-d834-41b2-8694-2267f7faf9c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:25.699920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d56c80-d834-41b2-8694-2267f7faf9c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:25.703219 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d56c80-d834-41b2-8694-2267f7faf9c2 service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:25.721935 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af32544-d012-46c4-86dc-c3b6c66dd8cc None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:25.724721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-2af32544-d012-46c4-86dc-c3b6c66dd8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:25.726046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 988/1983] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:25.827567 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af32544-d012-46c4-86dc-c3b6c66dd8cc None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:25.833675 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616881bc-48e2-4794-9800-ec2a08d36ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:25.834873 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616881bc-48e2-4794-9800-ec2a08d36ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:25.835167 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616881bc-48e2-4794-9800-ec2a08d36ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:25.835505 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616881bc-48e2-4794-9800-ec2a08d36ea6 admin admin] PATH_INFO: `/v3/projects/149ee4cf8d20400c8532289d783ea636` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:25.836906 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616881bc-48e2-4794-9800-ec2a08d36ea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=149ee4cf8d20400c8532289d783ea636)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:25.839035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616881bc-48e2-4794-9800-ec2a08d36ea6 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:26.048937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-616881bc-48e2-4794-9800-ec2a08d36ea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '149ee4cf8d20400c8532289d783ea636', 'request_id': 'req-616881bc-48e2-4794-9800-ec2a08d36ea6'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:26.153817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-616881bc-48e2-4794-9800-ec2a08d36ea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '149ee4cf8d20400c8532289d783ea636', 'request_id': 'req-616881bc-48e2-4794-9800-ec2a08d36ea6'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:26.166015 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-616881bc-48e2-4794-9800-ec2a08d36ea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '149ee4cf8d20400c8532289d783ea636', 'request_id': 'req-616881bc-48e2-4794-9800-ec2a08d36ea6'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:26.179729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-616881bc-48e2-4794-9800-ec2a08d36ea6 admin admin] The token cache is being invalidate because project 149ee4cf8d20400c8532289d783ea636 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:53:26.180378 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-616881bc-48e2-4794-9800-ec2a08d36ea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:26.181113 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-616881bc-48e2-4794-9800-ec2a08d36ea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:26.182985 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 996/1984] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:53:25 2022] DELETE /identity/v3/projects/149ee4cf8d20400c8532289d783ea636 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:26.191313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f0c4c-27aa-4ab4-b7ea-b916b8e9ce81 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:26.191674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f0c4c-27aa-4ab4-b7ea-b916b8e9ce81 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:26.192126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f0c4c-27aa-4ab4-b7ea-b916b8e9ce81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:26.223972 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-c19f0c4c-27aa-4ab4-b7ea-b916b8e9ce81 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:26.225574 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-c19f0c4c-27aa-4ab4-b7ea-b916b8e9ce81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:26.312461 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 989/1985] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:26.324823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84756fd8-eb46-48e0-bceb-ef0d0fc07fe4 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:26.325432 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84756fd8-eb46-48e0-bceb-ef0d0fc07fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:26.325814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84756fd8-eb46-48e0-bceb-ef0d0fc07fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:26.351881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-84756fd8-eb46-48e0-bceb-ef0d0fc07fe4 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:26.353176 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-84756fd8-eb46-48e0-bceb-ef0d0fc07fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:26.361888 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 997/1986] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:26.367338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f0c4c-27aa-4ab4-b7ea-b916b8e9ce81 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:26.385658 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f0c4c-27aa-4ab4-b7ea-b916b8e9ce81 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:26.391467 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:26.397171 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9bf38-042a-449f-af5a-9753d93c45a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:26.398762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9bf38-042a-449f-af5a-9753d93c45a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:26.399780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9bf38-042a-449f-af5a-9753d93c45a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:26.400456 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9bf38-042a-449f-af5a-9753d93c45a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:26.401560 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9bf38-042a-449f-af5a-9753d93c45a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:53:26.402076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9bf38-042a-449f-af5a-9753d93c45a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:26.405205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9bf38-042a-449f-af5a-9753d93c45a5 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:26.446730 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 990/1987] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:53:26.451344 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84756fd8-eb46-48e0-bceb-ef0d0fc07fe4 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:26.469572 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84756fd8-eb46-48e0-bceb-ef0d0fc07fe4 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:26.479058 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e951a-4513-4492-bd6b-ab9d12628239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:26.480055 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e951a-4513-4492-bd6b-ab9d12628239 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:26.480543 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e951a-4513-4492-bd6b-ab9d12628239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:26.480996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e951a-4513-4492-bd6b-ab9d12628239 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:26.482017 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e951a-4513-4492-bd6b-ab9d12628239 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:26.485538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e951a-4513-4492-bd6b-ab9d12628239 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:26.649933 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 998/1988] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:53:26 2022] POST /identity/v3/projects => generated 401 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:26.654522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9bf38-042a-449f-af5a-9753d93c45a5 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:26.666517 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9bf38-042a-449f-af5a-9753d93c45a5 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:26.775093 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605afb99-91a1-4f34-aa7d-92d7311b8881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:26.776551 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605afb99-91a1-4f34-aa7d-92d7311b8881 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:26.777458 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605afb99-91a1-4f34-aa7d-92d7311b8881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:26.777978 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605afb99-91a1-4f34-aa7d-92d7311b8881 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:26.779775 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605afb99-91a1-4f34-aa7d-92d7311b8881 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:26.803553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605afb99-91a1-4f34-aa7d-92d7311b8881 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:26.871036 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:26.911804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 991/1989] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:53:26 2022] POST /identity/v3/users => generated 301 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:53:26.917912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e951a-4513-4492-bd6b-ab9d12628239 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:26.941129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e951a-4513-4492-bd6b-ab9d12628239 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:26.954089 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4379cb7-e697-433d-8585-7182efe65bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:26.955387 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4379cb7-e697-433d-8585-7182efe65bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:26.955910 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4379cb7-e697-433d-8585-7182efe65bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:26.956419 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4379cb7-e697-433d-8585-7182efe65bc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:26.961943 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4379cb7-e697-433d-8585-7182efe65bc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:26.967074 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4379cb7-e697-433d-8585-7182efe65bc7 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:27.001668 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 999/1990] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:27.007401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605afb99-91a1-4f34-aa7d-92d7311b8881 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:27.030158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605afb99-91a1-4f34-aa7d-92d7311b8881 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:27.036786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf510e2-fa60-4c55-b392-ad2f257ec0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:27.040703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf510e2-fa60-4c55-b392-ad2f257ec0c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:27.040703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf510e2-fa60-4c55-b392-ad2f257ec0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:27.040703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf510e2-fa60-4c55-b392-ad2f257ec0c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:27.040703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf510e2-fa60-4c55-b392-ad2f257ec0c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:27.040703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf510e2-fa60-4c55-b392-ad2f257ec0c6 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:27.061948 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 992/1991] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:27.068390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4379cb7-e697-433d-8585-7182efe65bc7 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:27.085638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4379cb7-e697-433d-8585-7182efe65bc7 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:27.093878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd526b-62ca-4a27-9f8c-7c513c6bc6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:27.095027 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd526b-62ca-4a27-9f8c-7c513c6bc6e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:27.095510 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd526b-62ca-4a27-9f8c-7c513c6bc6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:27.095981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd526b-62ca-4a27-9f8c-7c513c6bc6e5 admin admin] PATH_INFO: `/v3/projects/9ac2e7119f2146bfad5e3fbb2306f99a/users/26605bb0170f49d5ac4c2e021d3c911a/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:27.126007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd526b-62ca-4a27-9f8c-7c513c6bc6e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ac2e7119f2146bfad5e3fbb2306f99a, user_id=26605bb0170f49d5ac4c2e021d3c911a, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:27.129882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd526b-62ca-4a27-9f8c-7c513c6bc6e5 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:27.266995 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1000/1992] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:27 2022] PUT /identity/v3/projects/9ac2e7119f2146bfad5e3fbb2306f99a/users/26605bb0170f49d5ac4c2e021d3c911a/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:27.282043 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23f643-094a-4137-94ad-0f181fad7b00 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:27.282385 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23f643-094a-4137-94ad-0f181fad7b00 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:27.282871 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23f643-094a-4137-94ad-0f181fad7b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:27.322847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-aa23f643-094a-4137-94ad-0f181fad7b00 None None] MFA Rules not processed for user `26605bb0170f49d5ac4c2e021d3c911a`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:27.324267 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-aa23f643-094a-4137-94ad-0f181fad7b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:27.446631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 993/1993] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:27 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:27.453913 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd526b-62ca-4a27-9f8c-7c513c6bc6e5 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:27.578771 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd526b-62ca-4a27-9f8c-7c513c6bc6e5 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:27.585024 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e84bea-ef35-4300-a251-9d2b883b6c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:27.585754 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e84bea-ef35-4300-a251-9d2b883b6c55 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:27.585969 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e84bea-ef35-4300-a251-9d2b883b6c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:27.586216 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e84bea-ef35-4300-a251-9d2b883b6c55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:27.587102 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e84bea-ef35-4300-a251-9d2b883b6c55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:27.589881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e84bea-ef35-4300-a251-9d2b883b6c55 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:27.824433 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1001/1994] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:53:27 2022] POST /identity/v3/projects => generated 399 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:27.826233 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23f643-094a-4137-94ad-0f181fad7b00 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:27.839786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23f643-094a-4137-94ad-0f181fad7b00 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:27.988060 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06374afc-5bc9-45df-8092-80b65a696a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:27.998601 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06374afc-5bc9-45df-8092-80b65a696a79 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:27.998601 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06374afc-5bc9-45df-8092-80b65a696a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:27.998601 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06374afc-5bc9-45df-8092-80b65a696a79 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:27.998601 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:28.003763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06374afc-5bc9-45df-8092-80b65a696a79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:28.005344 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06374afc-5bc9-45df-8092-80b65a696a79 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:28.150496 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 994/1995] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:53:27 2022] POST /identity/v3/users => generated 306 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:28.155906 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e84bea-ef35-4300-a251-9d2b883b6c55 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:28.179640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e84bea-ef35-4300-a251-9d2b883b6c55 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:28.188647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac01227-b366-409a-91b3-0c39b8116820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:28.189617 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac01227-b366-409a-91b3-0c39b8116820 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:28.190181 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac01227-b366-409a-91b3-0c39b8116820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:28.191151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac01227-b366-409a-91b3-0c39b8116820 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:28.193112 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac01227-b366-409a-91b3-0c39b8116820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:28.196379 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac01227-b366-409a-91b3-0c39b8116820 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:28.231963 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1002/1996] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:28.244426 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06374afc-5bc9-45df-8092-80b65a696a79 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:28.256036 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06374afc-5bc9-45df-8092-80b65a696a79 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:28.263522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb70ebd-7e00-445d-9df2-2ac5330f4694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:28.264505 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb70ebd-7e00-445d-9df2-2ac5330f4694 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:28.264999 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb70ebd-7e00-445d-9df2-2ac5330f4694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:28.265324 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb70ebd-7e00-445d-9df2-2ac5330f4694 admin admin] PATH_INFO: `/v3/projects/2a8eb30e5dc24c5393e123961df113e9/users/d9bcf6e996f14eb0a81fbf07dbec2257/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:28.269074 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:28.307393 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb70ebd-7e00-445d-9df2-2ac5330f4694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a8eb30e5dc24c5393e123961df113e9, user_id=d9bcf6e996f14eb0a81fbf07dbec2257, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:28.313399 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb70ebd-7e00-445d-9df2-2ac5330f4694 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:28.449001 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 995/1997] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:28 2022] PUT /identity/v3/projects/2a8eb30e5dc24c5393e123961df113e9/users/d9bcf6e996f14eb0a81fbf07dbec2257/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:28.453625 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac01227-b366-409a-91b3-0c39b8116820 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:28.471279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac01227-b366-409a-91b3-0c39b8116820 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:28.556815 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27314ba3-6eec-4040-bbe6-b58fe6b75f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:28.557452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27314ba3-6eec-4040-bbe6-b58fe6b75f68 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:28.557734 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27314ba3-6eec-4040-bbe6-b58fe6b75f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:28.558035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27314ba3-6eec-4040-bbe6-b58fe6b75f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:28.558680 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27314ba3-6eec-4040-bbe6-b58fe6b75f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:28.560357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27314ba3-6eec-4040-bbe6-b58fe6b75f68 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:28.576372 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1003/1998] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:28.581021 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb70ebd-7e00-445d-9df2-2ac5330f4694 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:28.593306 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb70ebd-7e00-445d-9df2-2ac5330f4694 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:28.598940 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9fc71-a950-427d-8886-09acfde3827b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:28.600733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9fc71-a950-427d-8886-09acfde3827b admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:28.600733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9fc71-a950-427d-8886-09acfde3827b admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:28.600733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9fc71-a950-427d-8886-09acfde3827b admin admin] PATH_INFO: `/v3/projects/2a8eb30e5dc24c5393e123961df113e9/users/d9bcf6e996f14eb0a81fbf07dbec2257/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:28.602708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9fc71-a950-427d-8886-09acfde3827b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a8eb30e5dc24c5393e123961df113e9, user_id=d9bcf6e996f14eb0a81fbf07dbec2257, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:28.605451 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9fc71-a950-427d-8886-09acfde3827b admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:28.618861 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 996/1999] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:28 2022] PUT /identity/v3/projects/2a8eb30e5dc24c5393e123961df113e9/users/d9bcf6e996f14eb0a81fbf07dbec2257/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:28.630523 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e88756-3aa1-42d6-bdcc-6ab6897665e6 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:28.630893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e88756-3aa1-42d6-bdcc-6ab6897665e6 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:28.631169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e88756-3aa1-42d6-bdcc-6ab6897665e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:28.669721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-02e88756-3aa1-42d6-bdcc-6ab6897665e6 None None] MFA Rules not processed for user `d9bcf6e996f14eb0a81fbf07dbec2257`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:28.671975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-02e88756-3aa1-42d6-bdcc-6ab6897665e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:28.845818 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1004/2000] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:28 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:28.858526 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9fc71-a950-427d-8886-09acfde3827b admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:28.861239 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-73c9fc71-a950-427d-8886-09acfde3827b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:28.982283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9fc71-a950-427d-8886-09acfde3827b admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:28.990195 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7b4c6-e69f-4211-8be1-7a60a3646fc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:28.991138 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7b4c6-e69f-4211-8be1-7a60a3646fc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:28.991617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7b4c6-e69f-4211-8be1-7a60a3646fc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:28.992366 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7b4c6-e69f-4211-8be1-7a60a3646fc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:29.009392 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7b4c6-e69f-4211-8be1-7a60a3646fc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:29.014902 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7b4c6-e69f-4211-8be1-7a60a3646fc1 service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:29.028267 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 997/2001] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:28 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:33.585947 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e88756-3aa1-42d6-bdcc-6ab6897665e6 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:33.596998 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e88756-3aa1-42d6-bdcc-6ab6897665e6 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:33.601240 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d319b-6f24-418f-b155-6db090ddaee3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:33.601859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d319b-6f24-418f-b155-6db090ddaee3 service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:33.602365 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d319b-6f24-418f-b155-6db090ddaee3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:33.602708 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d319b-6f24-418f-b155-6db090ddaee3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:33.655890 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:33.675277 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7d319b-6f24-418f-b155-6db090ddaee3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:33.677784 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7d319b-6f24-418f-b155-6db090ddaee3 service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:33.690511 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1005/2002] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:33 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:35.257369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7b4c6-e69f-4211-8be1-7a60a3646fc1 service cinder] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:35.259695 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-54f7b4c6-e69f-4211-8be1-7a60a3646fc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:35.399746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7b4c6-e69f-4211-8be1-7a60a3646fc1 service cinder] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:35.399746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c4d62-4430-4c82-9c90-e720f8282d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:35.399746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c4d62-4430-4c82-9c90-e720f8282d45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:35.399746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c4d62-4430-4c82-9c90-e720f8282d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:35.401910 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c4d62-4430-4c82-9c90-e720f8282d45 admin admin] PATH_INFO: `/v3/users/38cf55e5e4a9466ab95b839f8208abee` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:35.401910 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c4d62-4430-4c82-9c90-e720f8282d45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38cf55e5e4a9466ab95b839f8208abee)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:35.401910 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c4d62-4430-4c82-9c90-e720f8282d45 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:35.579873 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:35.685163 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-905c4d62-4430-4c82-9c90-e720f8282d45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38cf55e5e4a9466ab95b839f8208abee'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:35.798983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-905c4d62-4430-4c82-9c90-e720f8282d45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38cf55e5e4a9466ab95b839f8208abee'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:35.800716 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-905c4d62-4430-4c82-9c90-e720f8282d45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38cf55e5e4a9466ab95b839f8208abee'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:35.809880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-905c4d62-4430-4c82-9c90-e720f8282d45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38cf55e5e4a9466ab95b839f8208abee'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:35.810717 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-905c4d62-4430-4c82-9c90-e720f8282d45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38cf55e5e4a9466ab95b839f8208abee'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:35.831780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 998/2003] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:53:35 2022] DELETE /identity/v3/users/38cf55e5e4a9466ab95b839f8208abee => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:35.837340 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334ead7-a0cd-410f-8880-8ea7b4d550b2 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:35.837852 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334ead7-a0cd-410f-8880-8ea7b4d550b2 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:35.840170 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334ead7-a0cd-410f-8880-8ea7b4d550b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:35.860051 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-0334ead7-a0cd-410f-8880-8ea7b4d550b2 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:35.861282 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-0334ead7-a0cd-410f-8880-8ea7b4d550b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:35.961164 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1006/2004] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:35.977625 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c4d62-4430-4c82-9c90-e720f8282d45 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:35.983834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-905c4d62-4430-4c82-9c90-e720f8282d45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:36.060424 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c4d62-4430-4c82-9c90-e720f8282d45 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:36.064825 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f8528-70bd-41ec-88db-44ca688c8998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:36.065679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f8528-70bd-41ec-88db-44ca688c8998 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:36.066044 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f8528-70bd-41ec-88db-44ca688c8998 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:36.066554 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f8528-70bd-41ec-88db-44ca688c8998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:36.086807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f8528-70bd-41ec-88db-44ca688c8998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:36.089758 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f8528-70bd-41ec-88db-44ca688c8998 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:36.109462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 999/2005] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:36.694660 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334ead7-a0cd-410f-8880-8ea7b4d550b2 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:36.696370 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-0334ead7-a0cd-410f-8880-8ea7b4d550b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:36.711743 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334ead7-a0cd-410f-8880-8ea7b4d550b2 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:36.725232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4130105-7c83-4760-aca2-809fae3d872b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:36.726211 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4130105-7c83-4760-aca2-809fae3d872b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:36.726409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4130105-7c83-4760-aca2-809fae3d872b admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:36.730722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4130105-7c83-4760-aca2-809fae3d872b admin admin] PATH_INFO: `/v3/projects/90e6b59b14594b7c9ba05790eb5fda3c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:36.730722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4130105-7c83-4760-aca2-809fae3d872b admin admin] RBAC: Authorizing `identity:delete_project(project_id=90e6b59b14594b7c9ba05790eb5fda3c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:36.730722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4130105-7c83-4760-aca2-809fae3d872b admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:36.939337 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-f4130105-7c83-4760-aca2-809fae3d872b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90e6b59b14594b7c9ba05790eb5fda3c', 'request_id': 'req-f4130105-7c83-4760-aca2-809fae3d872b'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:37.063409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-f4130105-7c83-4760-aca2-809fae3d872b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90e6b59b14594b7c9ba05790eb5fda3c', 'request_id': 'req-f4130105-7c83-4760-aca2-809fae3d872b'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:37.072544 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-f4130105-7c83-4760-aca2-809fae3d872b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90e6b59b14594b7c9ba05790eb5fda3c', 'request_id': 'req-f4130105-7c83-4760-aca2-809fae3d872b'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:37.087987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-f4130105-7c83-4760-aca2-809fae3d872b admin admin] The token cache is being invalidate because project 90e6b59b14594b7c9ba05790eb5fda3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:53:37.088402 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-f4130105-7c83-4760-aca2-809fae3d872b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:37.089252 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-f4130105-7c83-4760-aca2-809fae3d872b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:37.091102 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1007/2006] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:53:36 2022] DELETE /identity/v3/projects/90e6b59b14594b7c9ba05790eb5fda3c => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:37.095542 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f8528-70bd-41ec-88db-44ca688c8998 service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:37.107887 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f8528-70bd-41ec-88db-44ca688c8998 service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:37.182623 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735eeaaa-be11-4b0a-8364-2e37935e92bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:37.183407 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735eeaaa-be11-4b0a-8364-2e37935e92bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:37.183677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735eeaaa-be11-4b0a-8364-2e37935e92bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:37.183941 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735eeaaa-be11-4b0a-8364-2e37935e92bb admin admin] PATH_INFO: `/v3/users/c272f90d4baa4f22990ced81273e8484` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:37.185157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735eeaaa-be11-4b0a-8364-2e37935e92bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c272f90d4baa4f22990ced81273e8484)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:37.187236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735eeaaa-be11-4b0a-8364-2e37935e92bb admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:37.451975 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-735eeaaa-be11-4b0a-8364-2e37935e92bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c272f90d4baa4f22990ced81273e8484'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:37.455566 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:37.611274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-735eeaaa-be11-4b0a-8364-2e37935e92bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c272f90d4baa4f22990ced81273e8484'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:37.611935 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-735eeaaa-be11-4b0a-8364-2e37935e92bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c272f90d4baa4f22990ced81273e8484'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:37.620626 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-735eeaaa-be11-4b0a-8364-2e37935e92bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c272f90d4baa4f22990ced81273e8484'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:37.621367 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-735eeaaa-be11-4b0a-8364-2e37935e92bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c272f90d4baa4f22990ced81273e8484'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:37.641092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1000/2007] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:53:37 2022] DELETE /identity/v3/users/c272f90d4baa4f22990ced81273e8484 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:38.222968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4130105-7c83-4760-aca2-809fae3d872b admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:38.226419 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f4130105-7c83-4760-aca2-809fae3d872b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:38.251138 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4130105-7c83-4760-aca2-809fae3d872b admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:38.337178 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353db62c-9759-4136-b09a-b5cd4e5028f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:38.338242 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353db62c-9759-4136-b09a-b5cd4e5028f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:38.338604 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353db62c-9759-4136-b09a-b5cd4e5028f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:38.338940 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353db62c-9759-4136-b09a-b5cd4e5028f8 admin admin] PATH_INFO: `/v3/projects/bbebc6be7c4f4837aba22e1304dd2b9c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:38.340374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353db62c-9759-4136-b09a-b5cd4e5028f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbebc6be7c4f4837aba22e1304dd2b9c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:38.343028 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353db62c-9759-4136-b09a-b5cd4e5028f8 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:38.352065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:38.499130 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-353db62c-9759-4136-b09a-b5cd4e5028f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbebc6be7c4f4837aba22e1304dd2b9c', 'request_id': 'req-353db62c-9759-4136-b09a-b5cd4e5028f8'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:38.621873 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-353db62c-9759-4136-b09a-b5cd4e5028f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbebc6be7c4f4837aba22e1304dd2b9c', 'request_id': 'req-353db62c-9759-4136-b09a-b5cd4e5028f8'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:38.631201 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-353db62c-9759-4136-b09a-b5cd4e5028f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbebc6be7c4f4837aba22e1304dd2b9c', 'request_id': 'req-353db62c-9759-4136-b09a-b5cd4e5028f8'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:38.647787 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-353db62c-9759-4136-b09a-b5cd4e5028f8 admin admin] The token cache is being invalidate because project bbebc6be7c4f4837aba22e1304dd2b9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:53:38.648264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-353db62c-9759-4136-b09a-b5cd4e5028f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:38.649393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-353db62c-9759-4136-b09a-b5cd4e5028f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:38.651369 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1008/2008] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:53:38 2022] DELETE /identity/v3/projects/bbebc6be7c4f4837aba22e1304dd2b9c => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:38.657111 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34630dc9-aa87-4c5e-99cf-35fc96c2ccc0 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:38.658067 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34630dc9-aa87-4c5e-99cf-35fc96c2ccc0 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:38.658067 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34630dc9-aa87-4c5e-99cf-35fc96c2ccc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:38.691140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-34630dc9-aa87-4c5e-99cf-35fc96c2ccc0 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:38.694061 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-34630dc9-aa87-4c5e-99cf-35fc96c2ccc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:38.813261 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1001/2009] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:38.823273 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b500ab-f796-4660-8f6e-85e253fe2906 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:38.823702 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b500ab-f796-4660-8f6e-85e253fe2906 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:38.824090 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b500ab-f796-4660-8f6e-85e253fe2906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:38.847378 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-f8b500ab-f796-4660-8f6e-85e253fe2906 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:38.850026 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f8b500ab-f796-4660-8f6e-85e253fe2906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:38.860742 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1009/2010] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:38.867251 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34630dc9-aa87-4c5e-99cf-35fc96c2ccc0 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:38.879089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34630dc9-aa87-4c5e-99cf-35fc96c2ccc0 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:38.885212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd45b5-1feb-458b-a22d-a375ed7b63d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:38.885978 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd45b5-1feb-458b-a22d-a375ed7b63d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:38.886334 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd45b5-1feb-458b-a22d-a375ed7b63d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:38.886694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd45b5-1feb-458b-a22d-a375ed7b63d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:38.887482 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd45b5-1feb-458b-a22d-a375ed7b63d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:53:38.887835 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd45b5-1feb-458b-a22d-a375ed7b63d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:38.890074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd45b5-1feb-458b-a22d-a375ed7b63d0 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:38.908557 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1002/2011] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:53:38.915138 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b500ab-f796-4660-8f6e-85e253fe2906 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:38.927776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b500ab-f796-4660-8f6e-85e253fe2906 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:38.935889 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d28780-a0df-40db-80fb-e50976661d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:38.936936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d28780-a0df-40db-80fb-e50976661d3d admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:38.937422 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d28780-a0df-40db-80fb-e50976661d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:38.938420 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d28780-a0df-40db-80fb-e50976661d3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:38.939118 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d28780-a0df-40db-80fb-e50976661d3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:38.944288 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d28780-a0df-40db-80fb-e50976661d3d admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:39.052674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1010/2012] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:53:38 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:53:39.059226 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd45b5-1feb-458b-a22d-a375ed7b63d0 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:39.072916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd45b5-1feb-458b-a22d-a375ed7b63d0 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:39.171113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5089e-b302-4d19-9546-5b5355048832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:39.172152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5089e-b302-4d19-9546-5b5355048832 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:39.172613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5089e-b302-4d19-9546-5b5355048832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:39.173086 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5089e-b302-4d19-9546-5b5355048832 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:39.174225 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5089e-b302-4d19-9546-5b5355048832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:39.178116 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5089e-b302-4d19-9546-5b5355048832 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:39.281010 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1003/2013] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:53:39 2022] POST /identity/v3/users => generated 287 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:39.287026 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d28780-a0df-40db-80fb-e50976661d3d admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:39.302970 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d28780-a0df-40db-80fb-e50976661d3d admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:39.309986 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3091993-87c2-4d40-af1d-2050ae4268b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:39.310828 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3091993-87c2-4d40-af1d-2050ae4268b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:39.311296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3091993-87c2-4d40-af1d-2050ae4268b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:39.311712 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3091993-87c2-4d40-af1d-2050ae4268b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:39.312621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3091993-87c2-4d40-af1d-2050ae4268b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:39.315569 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3091993-87c2-4d40-af1d-2050ae4268b1 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:39.395583 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1011/2014] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:39.395965 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5089e-b302-4d19-9546-5b5355048832 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:39.408528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5089e-b302-4d19-9546-5b5355048832 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:39.415681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e712f-e556-44f8-9c50-39e826e9bbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:39.416303 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e712f-e556-44f8-9c50-39e826e9bbd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:39.416636 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e712f-e556-44f8-9c50-39e826e9bbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:39.416956 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e712f-e556-44f8-9c50-39e826e9bbd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:39.417686 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9e712f-e556-44f8-9c50-39e826e9bbd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:39.419577 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9e712f-e556-44f8-9c50-39e826e9bbd0 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:39.439400 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1004/2015] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:53:39.440762 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3091993-87c2-4d40-af1d-2050ae4268b1 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:39.455003 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3091993-87c2-4d40-af1d-2050ae4268b1 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:39.463003 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852579d-c545-4a72-baec-0659088b12ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:39.464116 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852579d-c545-4a72-baec-0659088b12ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:39.464570 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852579d-c545-4a72-baec-0659088b12ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:39.465023 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852579d-c545-4a72-baec-0659088b12ca admin admin] PATH_INFO: `/v3/projects/2aae583cbf0d4a399c56d98367d768a5/users/3d755f36d78b40b0aef85378606f003e/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:39.486901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852579d-c545-4a72-baec-0659088b12ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aae583cbf0d4a399c56d98367d768a5, user_id=3d755f36d78b40b0aef85378606f003e, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:39.490996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852579d-c545-4a72-baec-0659088b12ca admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:39.657363 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1012/2016] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:39 2022] PUT /identity/v3/projects/2aae583cbf0d4a399c56d98367d768a5/users/3d755f36d78b40b0aef85378606f003e/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:39.670633 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4261e0-7884-4406-a2e6-20358a09bf70 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:39.671275 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4261e0-7884-4406-a2e6-20358a09bf70 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:39.671576 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4261e0-7884-4406-a2e6-20358a09bf70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:39.720071 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-ed4261e0-7884-4406-a2e6-20358a09bf70 None None] MFA Rules not processed for user `3d755f36d78b40b0aef85378606f003e`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:39.722148 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ed4261e0-7884-4406-a2e6-20358a09bf70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:39.758000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:39.855366 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1005/2017] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:39.875721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852579d-c545-4a72-baec-0659088b12ca admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:39.878059 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b852579d-c545-4a72-baec-0659088b12ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:39.990906 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852579d-c545-4a72-baec-0659088b12ca admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:39.995791 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9af1e-ae9e-463e-8f62-b6a15f2ea1af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:39.996820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9af1e-ae9e-463e-8f62-b6a15f2ea1af service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:39.997211 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9af1e-ae9e-463e-8f62-b6a15f2ea1af service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:39.997636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9af1e-ae9e-463e-8f62-b6a15f2ea1af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:40.019915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9af1e-ae9e-463e-8f62-b6a15f2ea1af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:40.022621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9af1e-ae9e-463e-8f62-b6a15f2ea1af service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:40.042360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1013/2018] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:40.068076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4261e0-7884-4406-a2e6-20358a09bf70 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:40.151161 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4261e0-7884-4406-a2e6-20358a09bf70 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:40.156716 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa83ab-ea07-4938-aa9f-d2252f79ea97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:40.157403 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa83ab-ea07-4938-aa9f-d2252f79ea97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:40.157695 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa83ab-ea07-4938-aa9f-d2252f79ea97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:40.157951 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa83ab-ea07-4938-aa9f-d2252f79ea97 admin admin] PATH_INFO: `/v3/users/3d755f36d78b40b0aef85378606f003e` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:40.159299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa83ab-ea07-4938-aa9f-d2252f79ea97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d755f36d78b40b0aef85378606f003e)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:40.170911 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa83ab-ea07-4938-aa9f-d2252f79ea97 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:40.443290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-25aa83ab-ea07-4938-aa9f-d2252f79ea97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d755f36d78b40b0aef85378606f003e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:40.690503 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-25aa83ab-ea07-4938-aa9f-d2252f79ea97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d755f36d78b40b0aef85378606f003e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:40.692470 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-25aa83ab-ea07-4938-aa9f-d2252f79ea97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d755f36d78b40b0aef85378606f003e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:40.714797 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-25aa83ab-ea07-4938-aa9f-d2252f79ea97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d755f36d78b40b0aef85378606f003e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:40.716587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-25aa83ab-ea07-4938-aa9f-d2252f79ea97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d755f36d78b40b0aef85378606f003e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:40.765042 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1006/2019] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:53:40 2022] DELETE /identity/v3/users/3d755f36d78b40b0aef85378606f003e => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:40.768575 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5470c4-05f7-449e-9b98-8ad58ab6ac3d None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:40.769419 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5470c4-05f7-449e-9b98-8ad58ab6ac3d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:40.769419 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5470c4-05f7-449e-9b98-8ad58ab6ac3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:40.795873 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-6e5470c4-05f7-449e-9b98-8ad58ab6ac3d None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:40.797983 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-6e5470c4-05f7-449e-9b98-8ad58ab6ac3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:40.857693 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:40.906640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1014/2020] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:53:40.920811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa83ab-ea07-4938-aa9f-d2252f79ea97 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:40.925004 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-25aa83ab-ea07-4938-aa9f-d2252f79ea97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:41.009871 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:41.027285 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa83ab-ea07-4938-aa9f-d2252f79ea97 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:41.031937 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66bcd9d-48ef-4831-9ac0-a9087e78c975 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:41.033208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66bcd9d-48ef-4831-9ac0-a9087e78c975 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:41.033494 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66bcd9d-48ef-4831-9ac0-a9087e78c975 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:41.033867 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66bcd9d-48ef-4831-9ac0-a9087e78c975 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:41.054700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66bcd9d-48ef-4831-9ac0-a9087e78c975 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:41.056775 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66bcd9d-48ef-4831-9ac0-a9087e78c975 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:41.070303 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1007/2021] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:42.802345 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5470c4-05f7-449e-9b98-8ad58ab6ac3d None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:42.804013 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-6e5470c4-05f7-449e-9b98-8ad58ab6ac3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:42.817378 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5470c4-05f7-449e-9b98-8ad58ab6ac3d None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:42.822730 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f770670-8f44-4931-86a1-c2c30759bd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:42.823478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f770670-8f44-4931-86a1-c2c30759bd5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:42.823750 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f770670-8f44-4931-86a1-c2c30759bd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:42.824021 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f770670-8f44-4931-86a1-c2c30759bd5f admin admin] PATH_INFO: `/v3/projects/2aae583cbf0d4a399c56d98367d768a5` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:42.825639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f770670-8f44-4931-86a1-c2c30759bd5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aae583cbf0d4a399c56d98367d768a5)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:42.827767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f770670-8f44-4931-86a1-c2c30759bd5f admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:43.016071 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-1f770670-8f44-4931-86a1-c2c30759bd5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aae583cbf0d4a399c56d98367d768a5', 'request_id': 'req-1f770670-8f44-4931-86a1-c2c30759bd5f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:43.125476 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-1f770670-8f44-4931-86a1-c2c30759bd5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aae583cbf0d4a399c56d98367d768a5', 'request_id': 'req-1f770670-8f44-4931-86a1-c2c30759bd5f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:43.139368 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-1f770670-8f44-4931-86a1-c2c30759bd5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aae583cbf0d4a399c56d98367d768a5', 'request_id': 'req-1f770670-8f44-4931-86a1-c2c30759bd5f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:43.168183 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-1f770670-8f44-4931-86a1-c2c30759bd5f admin admin] The token cache is being invalidate because project 2aae583cbf0d4a399c56d98367d768a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:53:43.169074 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-1f770670-8f44-4931-86a1-c2c30759bd5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:43.170035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-1f770670-8f44-4931-86a1-c2c30759bd5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:43.177479 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1015/2022] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:53:42 2022] DELETE /identity/v3/projects/2aae583cbf0d4a399c56d98367d768a5 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:43.181407 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481263cd-d1d3-4d72-8df3-bcb6593c78dc None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:43.181407 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481263cd-d1d3-4d72-8df3-bcb6593c78dc None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:43.181407 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481263cd-d1d3-4d72-8df3-bcb6593c78dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:43.210176 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-481263cd-d1d3-4d72-8df3-bcb6593c78dc None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:43.212810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-481263cd-d1d3-4d72-8df3-bcb6593c78dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:43.337721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1008/2023] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:43.346883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a8f5f-bf54-44bc-821b-08a3a91036ef None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:43.347184 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a8f5f-bf54-44bc-821b-08a3a91036ef None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:43.347474 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a8f5f-bf54-44bc-821b-08a3a91036ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:43.374553 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-b79a8f5f-bf54-44bc-821b-08a3a91036ef None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:43.377018 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b79a8f5f-bf54-44bc-821b-08a3a91036ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:43.390494 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1016/2024] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:43.396991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481263cd-d1d3-4d72-8df3-bcb6593c78dc None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:43.412695 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481263cd-d1d3-4d72-8df3-bcb6593c78dc None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:43.418713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7de5d-5d75-48f7-84c3-b918890d9cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:43.420040 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7de5d-5d75-48f7-84c3-b918890d9cec admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:43.420498 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7de5d-5d75-48f7-84c3-b918890d9cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:43.420964 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7de5d-5d75-48f7-84c3-b918890d9cec admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:43.422291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7de5d-5d75-48f7-84c3-b918890d9cec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:53:43.422736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7de5d-5d75-48f7-84c3-b918890d9cec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:43.426899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7de5d-5d75-48f7-84c3-b918890d9cec admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:43.452274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1009/2025] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:53:43.458415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a8f5f-bf54-44bc-821b-08a3a91036ef None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:43.485121 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a8f5f-bf54-44bc-821b-08a3a91036ef None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:43.513664 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87081b77-7ee1-492f-b10f-c774861b7aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:43.515488 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87081b77-7ee1-492f-b10f-c774861b7aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:43.516359 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87081b77-7ee1-492f-b10f-c774861b7aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:43.517176 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87081b77-7ee1-492f-b10f-c774861b7aa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:43.518593 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87081b77-7ee1-492f-b10f-c774861b7aa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:43.522802 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87081b77-7ee1-492f-b10f-c774861b7aa0 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:43.650333 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1017/2026] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:53:43 2022] POST /identity/v3/projects => generated 375 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:43.652620 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7de5d-5d75-48f7-84c3-b918890d9cec admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:43.663914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7de5d-5d75-48f7-84c3-b918890d9cec admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:43.767085 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb299cc-031c-42da-b807-58c913ef78b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:43.786808 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb299cc-031c-42da-b807-58c913ef78b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:43.787592 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb299cc-031c-42da-b807-58c913ef78b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:43.788203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb299cc-031c-42da-b807-58c913ef78b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:43.789182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb299cc-031c-42da-b807-58c913ef78b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:43.792099 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb299cc-031c-42da-b807-58c913ef78b7 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:43.907164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1010/2027] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:53:43 2022] POST /identity/v3/users => generated 288 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:53:43.910463 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87081b77-7ee1-492f-b10f-c774861b7aa0 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:43.938821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87081b77-7ee1-492f-b10f-c774861b7aa0 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:43.945928 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0c527-9f42-4836-9874-b72056a0cd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:43.947912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0c527-9f42-4836-9874-b72056a0cd9b admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:43.948337 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0c527-9f42-4836-9874-b72056a0cd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:43.948705 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0c527-9f42-4836-9874-b72056a0cd9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:43.950667 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0c527-9f42-4836-9874-b72056a0cd9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:43.952769 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0c527-9f42-4836-9874-b72056a0cd9b admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:43.993549 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1018/2028] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:44.008280 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb299cc-031c-42da-b807-58c913ef78b7 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:44.060941 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb299cc-031c-42da-b807-58c913ef78b7 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:44.082427 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6006fef-79a8-427a-86b5-d97b1ad17785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:44.083323 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6006fef-79a8-427a-86b5-d97b1ad17785 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:44.083993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6006fef-79a8-427a-86b5-d97b1ad17785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:44.085396 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6006fef-79a8-427a-86b5-d97b1ad17785 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:44.088748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6006fef-79a8-427a-86b5-d97b1ad17785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:44.095531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6006fef-79a8-427a-86b5-d97b1ad17785 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:44.123624 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1011/2029] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:44.133327 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0c527-9f42-4836-9874-b72056a0cd9b admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:44.154734 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0c527-9f42-4836-9874-b72056a0cd9b admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:44.169972 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84608b-d2f6-48ca-9a52-c60148a30f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:44.169972 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84608b-d2f6-48ca-9a52-c60148a30f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:44.169972 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84608b-d2f6-48ca-9a52-c60148a30f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:44.169972 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84608b-d2f6-48ca-9a52-c60148a30f1a admin admin] PATH_INFO: `/v3/projects/ec2f75f8c92f4eca97ce6c701ef99d99/users/a176a19c3d924de5bd60d62c6c36839d/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:44.206372 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84608b-d2f6-48ca-9a52-c60148a30f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec2f75f8c92f4eca97ce6c701ef99d99, user_id=a176a19c3d924de5bd60d62c6c36839d, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:44.212727 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84608b-d2f6-48ca-9a52-c60148a30f1a admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:44.297492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6006fef-79a8-427a-86b5-d97b1ad17785 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:44.307490 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d6006fef-79a8-427a-86b5-d97b1ad17785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:44.315147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1019/2030] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:44 2022] PUT /identity/v3/projects/ec2f75f8c92f4eca97ce6c701ef99d99/users/a176a19c3d924de5bd60d62c6c36839d/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:44.335006 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6006fef-79a8-427a-86b5-d97b1ad17785 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:44.335329 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff263a-8947-4eb9-86e2-ea78d67bcd94 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:44.335707 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff263a-8947-4eb9-86e2-ea78d67bcd94 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:44.336165 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff263a-8947-4eb9-86e2-ea78d67bcd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:44.438779 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:44.444519 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-11ff263a-8947-4eb9-86e2-ea78d67bcd94 None None] MFA Rules not processed for user `a176a19c3d924de5bd60d62c6c36839d`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:44.452172 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-11ff263a-8947-4eb9-86e2-ea78d67bcd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:44.482628 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702ad20-88f2-4da7-9417-c3d4cef1841a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:44.482628 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702ad20-88f2-4da7-9417-c3d4cef1841a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:44.482628 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702ad20-88f2-4da7-9417-c3d4cef1841a admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:44.482628 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702ad20-88f2-4da7-9417-c3d4cef1841a admin admin] PATH_INFO: `/v3/users/26605bb0170f49d5ac4c2e021d3c911a` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:44.483324 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702ad20-88f2-4da7-9417-c3d4cef1841a admin admin] RBAC: Authorizing `identity:delete_user(user_id=26605bb0170f49d5ac4c2e021d3c911a)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:44.485875 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702ad20-88f2-4da7-9417-c3d4cef1841a admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:44.570785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:44.626790 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1020/2031] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:53:44.628789 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff263a-8947-4eb9-86e2-ea78d67bcd94 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:44.651543 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff263a-8947-4eb9-86e2-ea78d67bcd94 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:44.660097 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bff66f-2f5b-4874-a639-338a5cdc31df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:44.665143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bff66f-2f5b-4874-a639-338a5cdc31df admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:44.665143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bff66f-2f5b-4874-a639-338a5cdc31df admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:44.665143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bff66f-2f5b-4874-a639-338a5cdc31df admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:44.665143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bff66f-2f5b-4874-a639-338a5cdc31df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:44.667818 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bff66f-2f5b-4874-a639-338a5cdc31df admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:44.747193 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7702ad20-88f2-4da7-9417-c3d4cef1841a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26605bb0170f49d5ac4c2e021d3c911a'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:44.797445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1021/2032] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:53:44 2022] POST /identity/v3/projects => generated 375 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:53:44.801958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bff66f-2f5b-4874-a639-338a5cdc31df admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:44.856767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7702ad20-88f2-4da7-9417-c3d4cef1841a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26605bb0170f49d5ac4c2e021d3c911a'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:44.857536 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7702ad20-88f2-4da7-9417-c3d4cef1841a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26605bb0170f49d5ac4c2e021d3c911a'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:44.867859 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7702ad20-88f2-4da7-9417-c3d4cef1841a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26605bb0170f49d5ac4c2e021d3c911a'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:44.868642 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7702ad20-88f2-4da7-9417-c3d4cef1841a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26605bb0170f49d5ac4c2e021d3c911a'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:44.904722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1012/2033] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:53:44 2022] DELETE /identity/v3/users/26605bb0170f49d5ac4c2e021d3c911a => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:44.909970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219e4d2-310b-46db-995a-e93edf6ea439 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:44.912258 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219e4d2-310b-46db-995a-e93edf6ea439 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:44.912258 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219e4d2-310b-46db-995a-e93edf6ea439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:44.996173 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bff66f-2f5b-4874-a639-338a5cdc31df admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:45.009881 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-0219e4d2-310b-46db-995a-e93edf6ea439 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:45.015468 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-0219e4d2-310b-46db-995a-e93edf6ea439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:45.158042 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e27e8-25e1-4f71-932c-183c3dd3c104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:45.158042 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e27e8-25e1-4f71-932c-183c3dd3c104 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:45.164894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e27e8-25e1-4f71-932c-183c3dd3c104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:45.165694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e27e8-25e1-4f71-932c-183c3dd3c104 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:45.167253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e27e8-25e1-4f71-932c-183c3dd3c104 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:45.171411 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e27e8-25e1-4f71-932c-183c3dd3c104 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:45.277921 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1013/2034] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:45.308487 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219e4d2-310b-46db-995a-e93edf6ea439 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:45.312694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-0219e4d2-310b-46db-995a-e93edf6ea439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:45.328166 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1022/2035] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:53:44 2022] POST /identity/v3/users => generated 288 bytes in 527 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:45.334553 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e27e8-25e1-4f71-932c-183c3dd3c104 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:45.344116 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-bc5e27e8-25e1-4f71-932c-183c3dd3c104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:45.416432 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e27e8-25e1-4f71-932c-183c3dd3c104 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:45.435644 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816a20e-b6a6-4db8-82b7-4532c0737fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:45.437246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816a20e-b6a6-4db8-82b7-4532c0737fda admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:45.437965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816a20e-b6a6-4db8-82b7-4532c0737fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:45.439028 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816a20e-b6a6-4db8-82b7-4532c0737fda admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:45.440559 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816a20e-b6a6-4db8-82b7-4532c0737fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:45.444456 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816a20e-b6a6-4db8-82b7-4532c0737fda admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:45.484595 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219e4d2-310b-46db-995a-e93edf6ea439 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:45.496111 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67e0c5-c38c-44bc-9354-c97c9fc63e6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:45.497269 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67e0c5-c38c-44bc-9354-c97c9fc63e6f service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:45.498434 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67e0c5-c38c-44bc-9354-c97c9fc63e6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:45.499012 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67e0c5-c38c-44bc-9354-c97c9fc63e6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:45.513665 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1023/2036] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:45.516421 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8c6c5-2dea-4219-8b05-693429d61d9d None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:45.521030 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f67e0c5-c38c-44bc-9354-c97c9fc63e6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:45.523173 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8c6c5-2dea-4219-8b05-693429d61d9d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:45.524232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f67e0c5-c38c-44bc-9354-c97c9fc63e6f service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:45.526845 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8c6c5-2dea-4219-8b05-693429d61d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:45.567994 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1014/2037] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:45.570223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67e0c5-c38c-44bc-9354-c97c9fc63e6f service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:45.585329 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-ccb8c6c5-2dea-4219-8b05-693429d61d9d None None] MFA Rules not processed for user `00fdea0388e34b1eb5bfff49a3de5b57`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:45.589053 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ccb8c6c5-2dea-4219-8b05-693429d61d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:45.601043 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67e0c5-c38c-44bc-9354-c97c9fc63e6f service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:45.623780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c60d3-5908-4103-875c-c311b59ca548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:45.627178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c60d3-5908-4103-875c-c311b59ca548 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:45.628456 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c60d3-5908-4103-875c-c311b59ca548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:45.629980 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c60d3-5908-4103-875c-c311b59ca548 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:45.631537 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c60d3-5908-4103-875c-c311b59ca548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:45.635598 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c60d3-5908-4103-875c-c311b59ca548 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:45.646856 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:45.669662 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1015/2038] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:45.682434 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c60d3-5908-4103-875c-c311b59ca548 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:45.690945 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:45.727895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1024/2039] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:45.732730 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c60d3-5908-4103-875c-c311b59ca548 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:45.751815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab7d78-1e86-4b90-866c-4de41b3594fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:45.755254 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab7d78-1e86-4b90-866c-4de41b3594fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:45.755708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab7d78-1e86-4b90-866c-4de41b3594fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:45.755894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab7d78-1e86-4b90-866c-4de41b3594fc admin admin] PATH_INFO: `/v3/projects/5688be2745b448869bef7035fc950da7/users/699a2d5b488f4271ac38af00581b8f9e/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:45.789081 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdab7d78-1e86-4b90-866c-4de41b3594fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5688be2745b448869bef7035fc950da7, user_id=699a2d5b488f4271ac38af00581b8f9e, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:45.789081 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdab7d78-1e86-4b90-866c-4de41b3594fc admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:45.897336 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1016/2040] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:45 2022] PUT /identity/v3/projects/5688be2745b448869bef7035fc950da7/users/699a2d5b488f4271ac38af00581b8f9e/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:45.909467 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5cbbb-61b0-42db-90c1-56fd19755003 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:45.910959 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5cbbb-61b0-42db-90c1-56fd19755003 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:45.912162 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5cbbb-61b0-42db-90c1-56fd19755003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:46.027770 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-cae5cbbb-61b0-42db-90c1-56fd19755003 None None] MFA Rules not processed for user `699a2d5b488f4271ac38af00581b8f9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:46.033865 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-cae5cbbb-61b0-42db-90c1-56fd19755003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:46.202467 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1025/2041] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:46.206549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab7d78-1e86-4b90-866c-4de41b3594fc admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:46.304155 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab7d78-1e86-4b90-866c-4de41b3594fc admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:46.311619 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e959d-ba92-4f4d-a0ee-312e5f2e6f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:46.312843 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e959d-ba92-4f4d-a0ee-312e5f2e6f77 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:46.313318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e959d-ba92-4f4d-a0ee-312e5f2e6f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:46.313791 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e959d-ba92-4f4d-a0ee-312e5f2e6f77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:46.315329 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e959d-ba92-4f4d-a0ee-312e5f2e6f77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:46.318103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e959d-ba92-4f4d-a0ee-312e5f2e6f77 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:46.526023 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1017/2042] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:53:46 2022] POST /identity/v3/projects => generated 377 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:53:46.527645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5cbbb-61b0-42db-90c1-56fd19755003 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:46.576115 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5cbbb-61b0-42db-90c1-56fd19755003 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:46.684100 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e959d-ba92-4f4d-a0ee-312e5f2e6f77 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:46.685987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355bc6b2-741c-41e6-98f1-e092f3c539c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:46.687441 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-3d1e959d-ba92-4f4d-a0ee-312e5f2e6f77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:46.687640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bc6b2-741c-41e6-98f1-e092f3c539c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:46.687795 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bc6b2-741c-41e6-98f1-e092f3c539c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:46.688166 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bc6b2-741c-41e6-98f1-e092f3c539c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:46.692503 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355bc6b2-741c-41e6-98f1-e092f3c539c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:46.697164 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355bc6b2-741c-41e6-98f1-e092f3c539c6 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:46.734845 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e959d-ba92-4f4d-a0ee-312e5f2e6f77 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:46.744264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9d95f-7908-49d3-b271-0da0f0f53bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:46.745569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9d95f-7908-49d3-b271-0da0f0f53bc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:46.746325 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9d95f-7908-49d3-b271-0da0f0f53bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:46.747176 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9d95f-7908-49d3-b271-0da0f0f53bc5 admin admin] PATH_INFO: `/v3/projects/9ac2e7119f2146bfad5e3fbb2306f99a` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:46.749606 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9d95f-7908-49d3-b271-0da0f0f53bc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ac2e7119f2146bfad5e3fbb2306f99a)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:46.753228 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9d95f-7908-49d3-b271-0da0f0f53bc5 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:46.855497 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1026/2043] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:53:46 2022] POST /identity/v3/users => generated 295 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:46.859775 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355bc6b2-741c-41e6-98f1-e092f3c539c6 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:46.882293 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355bc6b2-741c-41e6-98f1-e092f3c539c6 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:46.891105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053de471-9461-49af-b839-359d58215624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:46.892196 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053de471-9461-49af-b839-359d58215624 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:46.892691 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053de471-9461-49af-b839-359d58215624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:46.893194 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053de471-9461-49af-b839-359d58215624 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:46.897366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053de471-9461-49af-b839-359d58215624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:46.901050 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053de471-9461-49af-b839-359d58215624 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:46.939910 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1027/2044] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:46.960069 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053de471-9461-49af-b839-359d58215624 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:46.980513 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053de471-9461-49af-b839-359d58215624 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:46.987192 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-34e9d95f-7908-49d3-b271-0da0f0f53bc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ac2e7119f2146bfad5e3fbb2306f99a', 'request_id': 'req-34e9d95f-7908-49d3-b271-0da0f0f53bc5'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:47.005884 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-34e9d95f-7908-49d3-b271-0da0f0f53bc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ac2e7119f2146bfad5e3fbb2306f99a', 'request_id': 'req-34e9d95f-7908-49d3-b271-0da0f0f53bc5'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:47.032387 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-34e9d95f-7908-49d3-b271-0da0f0f53bc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ac2e7119f2146bfad5e3fbb2306f99a', 'request_id': 'req-34e9d95f-7908-49d3-b271-0da0f0f53bc5'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:47.123419 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3daeaae-3654-44ec-ba9d-93a97dc73fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:47.124311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daeaae-3654-44ec-ba9d-93a97dc73fbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:47.124583 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daeaae-3654-44ec-ba9d-93a97dc73fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:47.124861 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daeaae-3654-44ec-ba9d-93a97dc73fbb admin admin] PATH_INFO: `/v3/projects/2b586d4b3b5a4739b6d81e7ed0e62fc6/users/06b1df855c10450f973fbf4c4c08a520/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:47.129672 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:47.154122 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3daeaae-3654-44ec-ba9d-93a97dc73fbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b586d4b3b5a4739b6d81e7ed0e62fc6, user_id=06b1df855c10450f973fbf4c4c08a520, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:47.160743 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3daeaae-3654-44ec-ba9d-93a97dc73fbb admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:47.168216 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-34e9d95f-7908-49d3-b271-0da0f0f53bc5 admin admin] The token cache is being invalidate because project 9ac2e7119f2146bfad5e3fbb2306f99a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:53:47.169992 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-34e9d95f-7908-49d3-b271-0da0f0f53bc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:47.171661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-34e9d95f-7908-49d3-b271-0da0f0f53bc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:47.177447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1018/2045] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:53:46 2022] DELETE /identity/v3/projects/9ac2e7119f2146bfad5e3fbb2306f99a => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:47.181792 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9d95f-7908-49d3-b271-0da0f0f53bc5 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:47.183464 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-34e9d95f-7908-49d3-b271-0da0f0f53bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:47.201791 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9d95f-7908-49d3-b271-0da0f0f53bc5 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:47.213706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65907521-c4d1-4cd7-90b6-9dc3f114362e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:47.213706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65907521-c4d1-4cd7-90b6-9dc3f114362e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:47.213706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65907521-c4d1-4cd7-90b6-9dc3f114362e admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:47.213706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65907521-c4d1-4cd7-90b6-9dc3f114362e admin admin] PATH_INFO: `/v3/users/d9bcf6e996f14eb0a81fbf07dbec2257` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:47.214345 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65907521-c4d1-4cd7-90b6-9dc3f114362e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9bcf6e996f14eb0a81fbf07dbec2257)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:47.214345 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65907521-c4d1-4cd7-90b6-9dc3f114362e admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:47.243669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1028/2046] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:46 2022] PUT /identity/v3/projects/2b586d4b3b5a4739b6d81e7ed0e62fc6/users/06b1df855c10450f973fbf4c4c08a520/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:47.249941 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3daeaae-3654-44ec-ba9d-93a97dc73fbb admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:47.252935 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-a3daeaae-3654-44ec-ba9d-93a97dc73fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:47.407299 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3daeaae-3654-44ec-ba9d-93a97dc73fbb admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:47.417100 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb14ad2-c748-4fa4-9b97-3cdb6d8df2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:47.418383 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb14ad2-c748-4fa4-9b97-3cdb6d8df2b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:47.418559 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb14ad2-c748-4fa4-9b97-3cdb6d8df2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:47.419016 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb14ad2-c748-4fa4-9b97-3cdb6d8df2b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:47.420265 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb14ad2-c748-4fa4-9b97-3cdb6d8df2b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:47.422638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb14ad2-c748-4fa4-9b97-3cdb6d8df2b7 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:47.446456 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1029/2047] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:53:47.453630 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb14ad2-c748-4fa4-9b97-3cdb6d8df2b7 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:47.463587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=85,keystone:INSERT=3,keystone:DELETE=12 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:47.473484 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb14ad2-c748-4fa4-9b97-3cdb6d8df2b7 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:47.484827 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-65907521-c4d1-4cd7-90b6-9dc3f114362e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9bcf6e996f14eb0a81fbf07dbec2257'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:47.485593 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf134e-17fe-4af2-9443-5821b0810df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:47.486873 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf134e-17fe-4af2-9443-5821b0810df1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:47.495453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf134e-17fe-4af2-9443-5821b0810df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:47.495453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf134e-17fe-4af2-9443-5821b0810df1 admin admin] PATH_INFO: `/v3/projects/2b586d4b3b5a4739b6d81e7ed0e62fc6/users/06b1df855c10450f973fbf4c4c08a520/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:47.495453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcf134e-17fe-4af2-9443-5821b0810df1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b586d4b3b5a4739b6d81e7ed0e62fc6, user_id=06b1df855c10450f973fbf4c4c08a520, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:47.498216 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcf134e-17fe-4af2-9443-5821b0810df1 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:47.528248 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1030/2048] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:47 2022] PUT /identity/v3/projects/2b586d4b3b5a4739b6d81e7ed0e62fc6/users/06b1df855c10450f973fbf4c4c08a520/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:47.547931 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44feb9-1df3-45f8-a4ce-9f383f3a9539 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:47.548419 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44feb9-1df3-45f8-a4ce-9f383f3a9539 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:47.548898 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44feb9-1df3-45f8-a4ce-9f383f3a9539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:47.601324 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-65907521-c4d1-4cd7-90b6-9dc3f114362e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9bcf6e996f14eb0a81fbf07dbec2257'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:47.602419 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-65907521-c4d1-4cd7-90b6-9dc3f114362e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9bcf6e996f14eb0a81fbf07dbec2257'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:47.603061 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-6a44feb9-1df3-45f8-a4ce-9f383f3a9539 None None] MFA Rules not processed for user `06b1df855c10450f973fbf4c4c08a520`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:47.604832 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-6a44feb9-1df3-45f8-a4ce-9f383f3a9539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:47.619074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-65907521-c4d1-4cd7-90b6-9dc3f114362e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9bcf6e996f14eb0a81fbf07dbec2257'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:47.620596 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-65907521-c4d1-4cd7-90b6-9dc3f114362e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9bcf6e996f14eb0a81fbf07dbec2257'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:47.672233 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1019/2049] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:53:47 2022] DELETE /identity/v3/users/d9bcf6e996f14eb0a81fbf07dbec2257 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:47.765297 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1031/2050] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:47 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:53:47.787945 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65907521-c4d1-4cd7-90b6-9dc3f114362e admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:47.792822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-65907521-c4d1-4cd7-90b6-9dc3f114362e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:48.026903 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65907521-c4d1-4cd7-90b6-9dc3f114362e admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:48.036262 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc2093-98dd-4cfa-9ed3-cdd237614345 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:48.037375 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc2093-98dd-4cfa-9ed3-cdd237614345 service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:48.038042 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc2093-98dd-4cfa-9ed3-cdd237614345 service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:48.038318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc2093-98dd-4cfa-9ed3-cdd237614345 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:48.040887 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-b7dc2093-98dd-4cfa-9ed3-cdd237614345 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:48.138549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dc2093-98dd-4cfa-9ed3-cdd237614345 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:48.138549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dc2093-98dd-4cfa-9ed3-cdd237614345 service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:48.162495 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1020/2051] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:53:48.464139 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1250b5c-6939-4b0c-8d84-d5b57d2c0deb None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:48.464950 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1250b5c-6939-4b0c-8d84-d5b57d2c0deb None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:48.467176 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1250b5c-6939-4b0c-8d84-d5b57d2c0deb None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:48.474256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1032/2052] 10.222.0.13 () {54 vars in 921 bytes} [Mon Aug 8 16:53:48 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:53:48.511964 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d51d72-bb2c-4db4-958f-6d7bfb9b77b0 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:48.513558 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d51d72-bb2c-4db4-958f-6d7bfb9b77b0 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:48.514012 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d51d72-bb2c-4db4-958f-6d7bfb9b77b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:48.544189 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-53d51d72-bb2c-4db4-958f-6d7bfb9b77b0 None None] MFA Rules not processed for user `6e74dbb121d749749cf5f9d9d7221008`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:48.545446 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-53d51d72-bb2c-4db4-958f-6d7bfb9b77b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:48.643110 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1021/2053] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Aug 8 16:53:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:48.658814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1250b5c-6939-4b0c-8d84-d5b57d2c0deb None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:48.661465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f1250b5c-6939-4b0c-8d84-d5b57d2c0deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:48.685607 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1250b5c-6939-4b0c-8d84-d5b57d2c0deb None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:48.696322 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205984cd-5b86-42ac-8aab-97d3ebf0ef61 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e74dbb121d749749cf5f9d9d7221008', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:48.697364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205984cd-5b86-42ac-8aab-97d3ebf0ef61 service placement] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:48.697866 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205984cd-5b86-42ac-8aab-97d3ebf0ef61 service placement] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:48.698331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205984cd-5b86-42ac-8aab-97d3ebf0ef61 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:48.718433 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205984cd-5b86-42ac-8aab-97d3ebf0ef61 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:48.720606 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205984cd-5b86-42ac-8aab-97d3ebf0ef61 service placement] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:48.751519 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1033/2054] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:48.761062 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d51d72-bb2c-4db4-958f-6d7bfb9b77b0 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:48.764952 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-53d51d72-bb2c-4db4-958f-6d7bfb9b77b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:48.790551 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:48.875921 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d51d72-bb2c-4db4-958f-6d7bfb9b77b0 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:48.881539 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e772d-be3f-487e-bdfc-799c75a516c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:48.882243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e772d-be3f-487e-bdfc-799c75a516c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:48.882692 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e772d-be3f-487e-bdfc-799c75a516c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:48.883102 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e772d-be3f-487e-bdfc-799c75a516c8 admin admin] PATH_INFO: `/v3/users/5891c6b280bd4ad68ef411759fcdec46` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:48.884362 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e772d-be3f-487e-bdfc-799c75a516c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5891c6b280bd4ad68ef411759fcdec46)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:48.886323 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e772d-be3f-487e-bdfc-799c75a516c8 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:48.926469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205984cd-5b86-42ac-8aab-97d3ebf0ef61 service placement] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:48.926469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-205984cd-5b86-42ac-8aab-97d3ebf0ef61 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:48.966465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205984cd-5b86-42ac-8aab-97d3ebf0ef61 service placement] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:48.975695 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa85c4-91d8-48cd-bf7a-e3e4a5875c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:48.977338 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa85c4-91d8-48cd-bf7a-e3e4a5875c34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:48.977689 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa85c4-91d8-48cd-bf7a-e3e4a5875c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:48.978120 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa85c4-91d8-48cd-bf7a-e3e4a5875c34 admin admin] PATH_INFO: `/v3/projects/2a8eb30e5dc24c5393e123961df113e9` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:48.980183 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa85c4-91d8-48cd-bf7a-e3e4a5875c34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a8eb30e5dc24c5393e123961df113e9)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:48.983511 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa85c4-91d8-48cd-bf7a-e3e4a5875c34 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:49.180150 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:49.198802 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-b94e772d-be3f-487e-bdfc-799c75a516c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5891c6b280bd4ad68ef411759fcdec46'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:49.215164 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-81aa85c4-91d8-48cd-bf7a-e3e4a5875c34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a8eb30e5dc24c5393e123961df113e9', 'request_id': 'req-81aa85c4-91d8-48cd-bf7a-e3e4a5875c34'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:49.291129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-b94e772d-be3f-487e-bdfc-799c75a516c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5891c6b280bd4ad68ef411759fcdec46'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:49.291987 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-b94e772d-be3f-487e-bdfc-799c75a516c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5891c6b280bd4ad68ef411759fcdec46'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:49.304581 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-b94e772d-be3f-487e-bdfc-799c75a516c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5891c6b280bd4ad68ef411759fcdec46'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:49.305361 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-b94e772d-be3f-487e-bdfc-799c75a516c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5891c6b280bd4ad68ef411759fcdec46'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:49.331634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1022/2055] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:53:48 2022] DELETE /identity/v3/users/5891c6b280bd4ad68ef411759fcdec46 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:49.370413 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-81aa85c4-91d8-48cd-bf7a-e3e4a5875c34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a8eb30e5dc24c5393e123961df113e9', 'request_id': 'req-81aa85c4-91d8-48cd-bf7a-e3e4a5875c34'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:49.380635 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-81aa85c4-91d8-48cd-bf7a-e3e4a5875c34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a8eb30e5dc24c5393e123961df113e9', 'request_id': 'req-81aa85c4-91d8-48cd-bf7a-e3e4a5875c34'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:49.404977 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-81aa85c4-91d8-48cd-bf7a-e3e4a5875c34 admin admin] The token cache is being invalidate because project 2a8eb30e5dc24c5393e123961df113e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:53:49.405277 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-81aa85c4-91d8-48cd-bf7a-e3e4a5875c34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:49.405956 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-81aa85c4-91d8-48cd-bf7a-e3e4a5875c34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:49.407639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1034/2056] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:53:48 2022] DELETE /identity/v3/projects/2a8eb30e5dc24c5393e123961df113e9 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:49.421029 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e912553-7c77-4a32-9973-f8296d389aee None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:49.421314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e912553-7c77-4a32-9973-f8296d389aee None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:49.421589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e912553-7c77-4a32-9973-f8296d389aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:49.444916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-6e912553-7c77-4a32-9973-f8296d389aee None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:49.446078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-6e912553-7c77-4a32-9973-f8296d389aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:49.524431 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1023/2057] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:53:49.536677 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a05ed-2c6a-413b-ad03-450377cd9973 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:49.537469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a05ed-2c6a-413b-ad03-450377cd9973 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:49.537469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a05ed-2c6a-413b-ad03-450377cd9973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:49.560109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-9e8a05ed-2c6a-413b-ad03-450377cd9973 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:49.561359 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-9e8a05ed-2c6a-413b-ad03-450377cd9973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:49.569642 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1035/2058] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:53:49.574978 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e912553-7c77-4a32-9973-f8296d389aee None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:49.585334 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e912553-7c77-4a32-9973-f8296d389aee None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:49.592110 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d4e52-65fb-443d-8387-4bfa1db504d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:49.592776 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d4e52-65fb-443d-8387-4bfa1db504d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:49.593054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d4e52-65fb-443d-8387-4bfa1db504d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:49.593323 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d4e52-65fb-443d-8387-4bfa1db504d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:49.593991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d4e52-65fb-443d-8387-4bfa1db504d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:53:49.594295 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d4e52-65fb-443d-8387-4bfa1db504d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:49.595964 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d4e52-65fb-443d-8387-4bfa1db504d1 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:49.612168 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1024/2059] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:53:49.618132 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a05ed-2c6a-413b-ad03-450377cd9973 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:49.629950 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a05ed-2c6a-413b-ad03-450377cd9973 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:49.636873 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad599d5f-6a1b-43c7-836a-c0918068c9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:49.637752 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad599d5f-6a1b-43c7-836a-c0918068c9a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:49.639113 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad599d5f-6a1b-43c7-836a-c0918068c9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:49.639113 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad599d5f-6a1b-43c7-836a-c0918068c9a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:49.639288 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad599d5f-6a1b-43c7-836a-c0918068c9a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:49.642736 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad599d5f-6a1b-43c7-836a-c0918068c9a0 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:49.643097 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d4e52-65fb-443d-8387-4bfa1db504d1 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:49.645498 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-9d2d4e52-65fb-443d-8387-4bfa1db504d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:49.673117 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d4e52-65fb-443d-8387-4bfa1db504d1 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:49.684276 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868a482-f23f-4069-a4fa-c944d4ed4ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:49.685545 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868a482-f23f-4069-a4fa-c944d4ed4ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:49.685896 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868a482-f23f-4069-a4fa-c944d4ed4ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:49.686353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868a482-f23f-4069-a4fa-c944d4ed4ca9 admin admin] PATH_INFO: `/v3/projects/e8beb878fb0d489cb0a2cf4a569bf475` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:49.688837 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868a482-f23f-4069-a4fa-c944d4ed4ca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8beb878fb0d489cb0a2cf4a569bf475)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:49.693159 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868a482-f23f-4069-a4fa-c944d4ed4ca9 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:49.718747 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:49.864106 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1036/2060] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:53:49 2022] POST /identity/v3/projects => generated 385 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:53:49.865907 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad599d5f-6a1b-43c7-836a-c0918068c9a0 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:49.881569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f868a482-f23f-4069-a4fa-c944d4ed4ca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8beb878fb0d489cb0a2cf4a569bf475', 'request_id': 'req-f868a482-f23f-4069-a4fa-c944d4ed4ca9'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:49.892169 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f868a482-f23f-4069-a4fa-c944d4ed4ca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8beb878fb0d489cb0a2cf4a569bf475', 'request_id': 'req-f868a482-f23f-4069-a4fa-c944d4ed4ca9'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:49.901937 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f868a482-f23f-4069-a4fa-c944d4ed4ca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8beb878fb0d489cb0a2cf4a569bf475', 'request_id': 'req-f868a482-f23f-4069-a4fa-c944d4ed4ca9'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:49.974165 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad599d5f-6a1b-43c7-836a-c0918068c9a0 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:49.981455 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc25f0-02a7-48b3-a77f-2cc4656fed13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:49.982204 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc25f0-02a7-48b3-a77f-2cc4656fed13 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:49.982653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc25f0-02a7-48b3-a77f-2cc4656fed13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:49.983212 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc25f0-02a7-48b3-a77f-2cc4656fed13 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:49.984078 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc25f0-02a7-48b3-a77f-2cc4656fed13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:49.986849 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc25f0-02a7-48b3-a77f-2cc4656fed13 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:50.045334 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f868a482-f23f-4069-a4fa-c944d4ed4ca9 admin admin] The token cache is being invalidate because project e8beb878fb0d489cb0a2cf4a569bf475 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:53:50.045639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f868a482-f23f-4069-a4fa-c944d4ed4ca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:50.046626 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f868a482-f23f-4069-a4fa-c944d4ed4ca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:50.048431 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1025/2061] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:53:49 2022] DELETE /identity/v3/projects/e8beb878fb0d489cb0a2cf4a569bf475 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:50.054783 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb583d82-85d0-48b9-8b93-540bbf187b67 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:50.055211 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb583d82-85d0-48b9-8b93-540bbf187b67 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:50.055591 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb583d82-85d0-48b9-8b93-540bbf187b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:50.079043 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-fb583d82-85d0-48b9-8b93-540bbf187b67 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:50.080541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-fb583d82-85d0-48b9-8b93-540bbf187b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:50.091393 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1026/2062] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:50.103610 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d9c0c-6db6-4ecd-b14a-874795f1232b None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:50.104052 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d9c0c-6db6-4ecd-b14a-874795f1232b None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:50.104420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d9c0c-6db6-4ecd-b14a-874795f1232b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:50.169024 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-497d9c0c-6db6-4ecd-b14a-874795f1232b None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:50.172607 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-497d9c0c-6db6-4ecd-b14a-874795f1232b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:50.179105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1037/2063] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:53:49 2022] POST /identity/v3/users => generated 293 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:50.180756 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc25f0-02a7-48b3-a77f-2cc4656fed13 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:50.188853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-9cfc25f0-02a7-48b3-a77f-2cc4656fed13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:50.198052 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1027/2064] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:50.203849 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d9c0c-6db6-4ecd-b14a-874795f1232b None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:50.216049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d9c0c-6db6-4ecd-b14a-874795f1232b None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:50.230488 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc25f0-02a7-48b3-a77f-2cc4656fed13 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:50.239829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ce0a8-74d5-4a76-8822-40c6b66e0c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:50.241302 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ce0a8-74d5-4a76-8822-40c6b66e0c6d admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:50.241688 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ce0a8-74d5-4a76-8822-40c6b66e0c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:50.242621 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ce0a8-74d5-4a76-8822-40c6b66e0c6d admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:50.242795 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855e409-40c9-47a6-926b-51c2ca0b0db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:50.242973 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ce0a8-74d5-4a76-8822-40c6b66e0c6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:53:50.243168 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ce0a8-74d5-4a76-8822-40c6b66e0c6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:50.243388 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855e409-40c9-47a6-926b-51c2ca0b0db0 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:50.244772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855e409-40c9-47a6-926b-51c2ca0b0db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:50.245265 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855e409-40c9-47a6-926b-51c2ca0b0db0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:50.246705 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855e409-40c9-47a6-926b-51c2ca0b0db0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:50.249230 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ce0a8-74d5-4a76-8822-40c6b66e0c6d admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:50.249914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855e409-40c9-47a6-926b-51c2ca0b0db0 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:50.268232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1028/2065] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:53:50.275060 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ce0a8-74d5-4a76-8822-40c6b66e0c6d admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:50.282614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1038/2066] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:50.285805 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855e409-40c9-47a6-926b-51c2ca0b0db0 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:50.291327 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ce0a8-74d5-4a76-8822-40c6b66e0c6d admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:50.298126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54668ee-69e4-4f48-8bb2-dd6ae4e57982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:50.298902 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54668ee-69e4-4f48-8bb2-dd6ae4e57982 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:50.299251 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54668ee-69e4-4f48-8bb2-dd6ae4e57982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:50.299778 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54668ee-69e4-4f48-8bb2-dd6ae4e57982 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:50.302836 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54668ee-69e4-4f48-8bb2-dd6ae4e57982 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:50.303223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54668ee-69e4-4f48-8bb2-dd6ae4e57982 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:50.307489 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855e409-40c9-47a6-926b-51c2ca0b0db0 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:50.316644 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08163315-3038-46ed-b1e3-b5b50bcfc6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:50.318062 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08163315-3038-46ed-b1e3-b5b50bcfc6d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:50.319412 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08163315-3038-46ed-b1e3-b5b50bcfc6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:50.319412 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08163315-3038-46ed-b1e3-b5b50bcfc6d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:50.320242 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08163315-3038-46ed-b1e3-b5b50bcfc6d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:50.323395 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08163315-3038-46ed-b1e3-b5b50bcfc6d8 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:50.345183 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1039/2067] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:53:50.354059 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08163315-3038-46ed-b1e3-b5b50bcfc6d8 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:50.398506 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08163315-3038-46ed-b1e3-b5b50bcfc6d8 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:50.413745 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe9c28-b8f2-4f4e-9a94-c697515dfacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:50.415250 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe9c28-b8f2-4f4e-9a94-c697515dfacf admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:50.419173 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe9c28-b8f2-4f4e-9a94-c697515dfacf admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:50.419173 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe9c28-b8f2-4f4e-9a94-c697515dfacf admin admin] PATH_INFO: `/v3/projects/5d85700a87244733ab9862d251c68e7b/users/fe834253b840419292082274f9842873/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:50.449138 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe9c28-b8f2-4f4e-9a94-c697515dfacf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d85700a87244733ab9862d251c68e7b, user_id=fe834253b840419292082274f9842873, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:50.454623 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe9c28-b8f2-4f4e-9a94-c697515dfacf admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:50.476616 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1029/2068] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:53:50 2022] POST /identity/v3/projects => generated 385 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:53:50.478245 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54668ee-69e4-4f48-8bb2-dd6ae4e57982 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:50.572050 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54668ee-69e4-4f48-8bb2-dd6ae4e57982 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:50.577359 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba7375-c9e7-4377-afac-934fbe142f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:50.577986 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba7375-c9e7-4377-afac-934fbe142f33 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:50.578278 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba7375-c9e7-4377-afac-934fbe142f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:50.578541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba7375-c9e7-4377-afac-934fbe142f33 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:50.579221 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba7375-c9e7-4377-afac-934fbe142f33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:50.581234 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba7375-c9e7-4377-afac-934fbe142f33 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:50.633778 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1040/2069] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:50 2022] PUT /identity/v3/projects/5d85700a87244733ab9862d251c68e7b/users/fe834253b840419292082274f9842873/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:50.638475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe9c28-b8f2-4f4e-9a94-c697515dfacf admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:50.640469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-3abe9c28-b8f2-4f4e-9a94-c697515dfacf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:50.688762 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:50.709824 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe9c28-b8f2-4f4e-9a94-c697515dfacf admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:50.717008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1030/2070] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:53:50 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:53:50.717008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33cc64a-1d6a-493c-9d03-d4ec0deb0e71 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:50.717008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33cc64a-1d6a-493c-9d03-d4ec0deb0e71 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:50.717008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33cc64a-1d6a-493c-9d03-d4ec0deb0e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:50.724964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31160f22-3c95-43f7-8040-b28069f25e7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:50.726244 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31160f22-3c95-43f7-8040-b28069f25e7d service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:50.728209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31160f22-3c95-43f7-8040-b28069f25e7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:50.728715 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31160f22-3c95-43f7-8040-b28069f25e7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:50.733132 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-31160f22-3c95-43f7-8040-b28069f25e7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:50.760524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-e33cc64a-1d6a-493c-9d03-d4ec0deb0e71 None None] MFA Rules not processed for user `fe834253b840419292082274f9842873`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:50.761758 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-e33cc64a-1d6a-493c-9d03-d4ec0deb0e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:50.798526 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:50.823357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31160f22-3c95-43f7-8040-b28069f25e7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:50.826193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31160f22-3c95-43f7-8040-b28069f25e7d service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:50.843705 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1041/2071] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:50.845256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31160f22-3c95-43f7-8040-b28069f25e7d service cinder] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:50.868427 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1031/2072] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:50.876063 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33cc64a-1d6a-493c-9d03-d4ec0deb0e71 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:51.004337 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31160f22-3c95-43f7-8040-b28069f25e7d service cinder] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:51.017460 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242731aa-d81b-4b8f-ae31-ec35b53d4368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:51.020257 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242731aa-d81b-4b8f-ae31-ec35b53d4368 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:51.023101 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242731aa-d81b-4b8f-ae31-ec35b53d4368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:51.023765 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242731aa-d81b-4b8f-ae31-ec35b53d4368 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:51.026481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242731aa-d81b-4b8f-ae31-ec35b53d4368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:51.035262 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242731aa-d81b-4b8f-ae31-ec35b53d4368 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:51.089599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1042/2073] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:51.091607 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33cc64a-1d6a-493c-9d03-d4ec0deb0e71 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:51.096248 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242731aa-d81b-4b8f-ae31-ec35b53d4368 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:51.102658 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4525f-df37-4a4d-9c17-016305cf2b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:51.103844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4525f-df37-4a4d-9c17-016305cf2b4a admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:51.104436 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4525f-df37-4a4d-9c17-016305cf2b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:51.105049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4525f-df37-4a4d-9c17-016305cf2b4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:51.108618 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4525f-df37-4a4d-9c17-016305cf2b4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:51.113558 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4525f-df37-4a4d-9c17-016305cf2b4a admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:51.120214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242731aa-d81b-4b8f-ae31-ec35b53d4368 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:51.130330 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da05bad-80d5-4020-8745-f572720380fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:51.131702 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da05bad-80d5-4020-8745-f572720380fb admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:51.132348 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da05bad-80d5-4020-8745-f572720380fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:51.132982 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da05bad-80d5-4020-8745-f572720380fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:51.135026 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da05bad-80d5-4020-8745-f572720380fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:51.138405 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da05bad-80d5-4020-8745-f572720380fb admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:51.173470 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1043/2074] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:53:51.185468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da05bad-80d5-4020-8745-f572720380fb admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:51.215276 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da05bad-80d5-4020-8745-f572720380fb admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:51.226257 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d335eb-07a4-4cac-a81d-e0fb8f2b5ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:51.227616 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d335eb-07a4-4cac-a81d-e0fb8f2b5ab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:51.228197 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d335eb-07a4-4cac-a81d-e0fb8f2b5ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:51.228771 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d335eb-07a4-4cac-a81d-e0fb8f2b5ab5 admin admin] PATH_INFO: `/v3/projects/fa4be8df3fe34d71a3c3d91995c1edae/users/d6d78cbc233a498db3a9268a4b93d737/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:51.275669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d335eb-07a4-4cac-a81d-e0fb8f2b5ab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa4be8df3fe34d71a3c3d91995c1edae, user_id=d6d78cbc233a498db3a9268a4b93d737, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:51.281677 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d335eb-07a4-4cac-a81d-e0fb8f2b5ab5 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:51.291014 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1032/2075] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:53:50 2022] POST /identity/v3/projects => generated 389 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:51.291014 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4525f-df37-4a4d-9c17-016305cf2b4a admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:51.395360 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4525f-df37-4a4d-9c17-016305cf2b4a admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:51.396750 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1044/2076] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:51 2022] PUT /identity/v3/projects/fa4be8df3fe34d71a3c3d91995c1edae/users/d6d78cbc233a498db3a9268a4b93d737/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:51.402390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151738d8-c97a-40eb-9184-0c91e5adba74 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:51.402700 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151738d8-c97a-40eb-9184-0c91e5adba74 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:51.402959 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151738d8-c97a-40eb-9184-0c91e5adba74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:51.424307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-151738d8-c97a-40eb-9184-0c91e5adba74 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:51.426118 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-151738d8-c97a-40eb-9184-0c91e5adba74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:51.487085 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c85b6-c608-4489-9e75-2c3557c71508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:51.488495 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c85b6-c608-4489-9e75-2c3557c71508 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:51.488901 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c85b6-c608-4489-9e75-2c3557c71508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:51.489291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c85b6-c608-4489-9e75-2c3557c71508 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:51.490007 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c85b6-c608-4489-9e75-2c3557c71508 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:51.492362 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c85b6-c608-4489-9e75-2c3557c71508 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:51.501346 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:51.513294 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:51.537683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1045/2077] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:53:51.549318 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151738d8-c97a-40eb-9184-0c91e5adba74 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:51.553404 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-151738d8-c97a-40eb-9184-0c91e5adba74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:51.635107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151738d8-c97a-40eb-9184-0c91e5adba74 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:51.647317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49dd8c-e064-44da-88cc-444d93ebc32b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:51.647967 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49dd8c-e064-44da-88cc-444d93ebc32b service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:51.648957 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49dd8c-e064-44da-88cc-444d93ebc32b service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:51.648957 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49dd8c-e064-44da-88cc-444d93ebc32b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:51.655071 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1033/2078] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:53:51 2022] POST /identity/v3/users => generated 301 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:51.656801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c85b6-c608-4489-9e75-2c3557c71508 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:51.666008 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49dd8c-e064-44da-88cc-444d93ebc32b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:51.668827 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49dd8c-e064-44da-88cc-444d93ebc32b service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:51.684831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c85b6-c608-4489-9e75-2c3557c71508 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:51.697860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564379c9-14da-48a3-9818-35e261e0bf84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:51.699718 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564379c9-14da-48a3-9818-35e261e0bf84 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:51.712639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564379c9-14da-48a3-9818-35e261e0bf84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:51.713215 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1046/2079] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:53:51.713363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564379c9-14da-48a3-9818-35e261e0bf84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:51.713363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564379c9-14da-48a3-9818-35e261e0bf84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:51.713363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564379c9-14da-48a3-9818-35e261e0bf84 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:51.729485 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1034/2080] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:51.734868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49dd8c-e064-44da-88cc-444d93ebc32b service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:51.749929 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49dd8c-e064-44da-88cc-444d93ebc32b service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:51.755985 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe02b6-a8a5-40ee-94d6-a90788a7541a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:51.756739 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe02b6-a8a5-40ee-94d6-a90788a7541a admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:51.757131 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe02b6-a8a5-40ee-94d6-a90788a7541a admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:51.757505 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe02b6-a8a5-40ee-94d6-a90788a7541a admin admin] PATH_INFO: `/v3/projects/9ce3bd673a6a488a95a3defe50f6d541/users/e1b268763c624098a3824e1b47a0cbb0/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:51.778528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe02b6-a8a5-40ee-94d6-a90788a7541a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ce3bd673a6a488a95a3defe50f6d541, user_id=e1b268763c624098a3824e1b47a0cbb0, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:51.781481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe02b6-a8a5-40ee-94d6-a90788a7541a admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:51.888130 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1047/2081] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:51 2022] PUT /identity/v3/projects/9ce3bd673a6a488a95a3defe50f6d541/users/e1b268763c624098a3824e1b47a0cbb0/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:51.892717 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564379c9-14da-48a3-9818-35e261e0bf84 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:51.904761 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564379c9-14da-48a3-9818-35e261e0bf84 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:51.988688 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060a040-e7be-46cb-b8e6-1f76ae092e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:51.989390 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060a040-e7be-46cb-b8e6-1f76ae092e8a admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:51.989666 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060a040-e7be-46cb-b8e6-1f76ae092e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:51.989943 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060a040-e7be-46cb-b8e6-1f76ae092e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:51.990659 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060a040-e7be-46cb-b8e6-1f76ae092e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:51.992295 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060a040-e7be-46cb-b8e6-1f76ae092e8a admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:52.004125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1035/2082] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:52.008925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe02b6-a8a5-40ee-94d6-a90788a7541a admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:52.021507 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe02b6-a8a5-40ee-94d6-a90788a7541a admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:52.026912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95cfdc4-b39b-499b-b529-7f6369ef60b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:52.027724 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cfdc4-b39b-499b-b529-7f6369ef60b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:52.028042 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cfdc4-b39b-499b-b529-7f6369ef60b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:52.028368 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cfdc4-b39b-499b-b529-7f6369ef60b0 admin admin] PATH_INFO: `/v3/projects/9ce3bd673a6a488a95a3defe50f6d541/users/e1b268763c624098a3824e1b47a0cbb0/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:52.030605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95cfdc4-b39b-499b-b529-7f6369ef60b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ce3bd673a6a488a95a3defe50f6d541, user_id=e1b268763c624098a3824e1b47a0cbb0, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:52.035123 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95cfdc4-b39b-499b-b529-7f6369ef60b0 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:52.055558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1048/2083] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:52 2022] PUT /identity/v3/projects/9ce3bd673a6a488a95a3defe50f6d541/users/e1b268763c624098a3824e1b47a0cbb0/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:52.068157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db97c7-6f66-4684-9c82-3be80a770c94 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:52.068510 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db97c7-6f66-4684-9c82-3be80a770c94 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:52.068821 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db97c7-6f66-4684-9c82-3be80a770c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:52.109369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-f5db97c7-6f66-4684-9c82-3be80a770c94 None None] MFA Rules not processed for user `e1b268763c624098a3824e1b47a0cbb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:52.111283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-f5db97c7-6f66-4684-9c82-3be80a770c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:52.265639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1036/2084] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:52 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:52.274450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95cfdc4-b39b-499b-b529-7f6369ef60b0 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:52.277240 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b95cfdc4-b39b-499b-b529-7f6369ef60b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:52.374475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95cfdc4-b39b-499b-b529-7f6369ef60b0 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:52.388500 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87223c4-d22b-423c-a0b2-334ae03a4cab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:52.390075 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87223c4-d22b-423c-a0b2-334ae03a4cab service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:52.390805 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87223c4-d22b-423c-a0b2-334ae03a4cab service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:52.391526 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87223c4-d22b-423c-a0b2-334ae03a4cab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:52.414459 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87223c4-d22b-423c-a0b2-334ae03a4cab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:52.418460 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87223c4-d22b-423c-a0b2-334ae03a4cab service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:52.468133 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1049/2085] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:52 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:52.711783 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db97c7-6f66-4684-9c82-3be80a770c94 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:52.798681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db97c7-6f66-4684-9c82-3be80a770c94 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:52.805540 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154df86-f086-4c38-9dbe-2d141071bca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:52.806225 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154df86-f086-4c38-9dbe-2d141071bca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:52.806534 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154df86-f086-4c38-9dbe-2d141071bca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:52.807008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154df86-f086-4c38-9dbe-2d141071bca0 admin admin] PATH_INFO: `/v3/users/fe834253b840419292082274f9842873` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:52.808364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b154df86-f086-4c38-9dbe-2d141071bca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe834253b840419292082274f9842873)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:52.810675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b154df86-f086-4c38-9dbe-2d141071bca0 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:52.812919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:53.070244 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87223c4-d22b-423c-a0b2-334ae03a4cab service cinder] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:53.081142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-b154df86-f086-4c38-9dbe-2d141071bca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe834253b840419292082274f9842873'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:53.081794 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87223c4-d22b-423c-a0b2-334ae03a4cab service cinder] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:53.087851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beaaa78-e5ec-4cce-ac62-c7ac9a4651d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:53.089643 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beaaa78-e5ec-4cce-ac62-c7ac9a4651d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:53.090121 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beaaa78-e5ec-4cce-ac62-c7ac9a4651d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:53.090389 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beaaa78-e5ec-4cce-ac62-c7ac9a4651d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:53.092426 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-9beaaa78-e5ec-4cce-ac62-c7ac9a4651d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:53.177691 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-b154df86-f086-4c38-9dbe-2d141071bca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe834253b840419292082274f9842873'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:53.178861 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-b154df86-f086-4c38-9dbe-2d141071bca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe834253b840419292082274f9842873'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:53.179276 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:53.182180 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beaaa78-e5ec-4cce-ac62-c7ac9a4651d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:53.184045 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beaaa78-e5ec-4cce-ac62-c7ac9a4651d9 service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:53.191654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-b154df86-f086-4c38-9dbe-2d141071bca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe834253b840419292082274f9842873'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:53.193332 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-b154df86-f086-4c38-9dbe-2d141071bca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe834253b840419292082274f9842873'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:53.199797 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1050/2086] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:53.229620 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1037/2087] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:53:52 2022] DELETE /identity/v3/users/fe834253b840419292082274f9842873 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:53.234560 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c81d29-18f4-47c2-b378-061134890cb6 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:53.234925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c81d29-18f4-47c2-b378-061134890cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:53.235355 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c81d29-18f4-47c2-b378-061134890cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:53.259592 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-90c81d29-18f4-47c2-b378-061134890cb6 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:53.261013 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-90c81d29-18f4-47c2-b378-061134890cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:53.358726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1051/2088] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:53.367911 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154df86-f086-4c38-9dbe-2d141071bca0 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:53.379129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-b154df86-f086-4c38-9dbe-2d141071bca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:53.511490 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154df86-f086-4c38-9dbe-2d141071bca0 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:53.522612 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd62ad-86b5-417f-8a3e-d62c315b21c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:53.526477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd62ad-86b5-417f-8a3e-d62c315b21c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:53.532060 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd62ad-86b5-417f-8a3e-d62c315b21c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:53.532536 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd62ad-86b5-417f-8a3e-d62c315b21c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:53.547217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd62ad-86b5-417f-8a3e-d62c315b21c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:53.548824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd62ad-86b5-417f-8a3e-d62c315b21c8 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:53.569066 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1038/2089] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:54.457240 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c81d29-18f4-47c2-b378-061134890cb6 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:54.464586 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-90c81d29-18f4-47c2-b378-061134890cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:54.479997 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c81d29-18f4-47c2-b378-061134890cb6 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:54.487752 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e7fae-1f35-4a05-a8a4-9f277c0439c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:54.488830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e7fae-1f35-4a05-a8a4-9f277c0439c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:54.489181 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e7fae-1f35-4a05-a8a4-9f277c0439c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:54.489691 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e7fae-1f35-4a05-a8a4-9f277c0439c1 admin admin] PATH_INFO: `/v3/projects/5d85700a87244733ab9862d251c68e7b` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:54.493048 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586e7fae-1f35-4a05-a8a4-9f277c0439c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d85700a87244733ab9862d251c68e7b)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:54.499037 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586e7fae-1f35-4a05-a8a4-9f277c0439c1 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:54.781382 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-586e7fae-1f35-4a05-a8a4-9f277c0439c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d85700a87244733ab9862d251c68e7b', 'request_id': 'req-586e7fae-1f35-4a05-a8a4-9f277c0439c1'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:54.903367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-586e7fae-1f35-4a05-a8a4-9f277c0439c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d85700a87244733ab9862d251c68e7b', 'request_id': 'req-586e7fae-1f35-4a05-a8a4-9f277c0439c1'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:54.913040 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-586e7fae-1f35-4a05-a8a4-9f277c0439c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d85700a87244733ab9862d251c68e7b', 'request_id': 'req-586e7fae-1f35-4a05-a8a4-9f277c0439c1'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:54.929948 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-586e7fae-1f35-4a05-a8a4-9f277c0439c1 admin admin] The token cache is being invalidate because project 5d85700a87244733ab9862d251c68e7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:53:54.930226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-586e7fae-1f35-4a05-a8a4-9f277c0439c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:54.931197 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-586e7fae-1f35-4a05-a8a4-9f277c0439c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:54.933267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1052/2090] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:53:54 2022] DELETE /identity/v3/projects/5d85700a87244733ab9862d251c68e7b => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:54.937842 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd62ad-86b5-417f-8a3e-d62c315b21c8 service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:54.955413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd62ad-86b5-417f-8a3e-d62c315b21c8 service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:55.100503 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc6947-6d52-47b3-b6a9-08eef07bdefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:55.101222 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc6947-6d52-47b3-b6a9-08eef07bdefb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:55.101494 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc6947-6d52-47b3-b6a9-08eef07bdefb admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:55.101757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc6947-6d52-47b3-b6a9-08eef07bdefb admin admin] PATH_INFO: `/v3/users/e1b268763c624098a3824e1b47a0cbb0` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:55.103124 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc6947-6d52-47b3-b6a9-08eef07bdefb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1b268763c624098a3824e1b47a0cbb0)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:55.105377 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc6947-6d52-47b3-b6a9-08eef07bdefb admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:55.149801 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:55.435452 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-01fc6947-6d52-47b3-b6a9-08eef07bdefb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1b268763c624098a3824e1b47a0cbb0'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:55.571183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-01fc6947-6d52-47b3-b6a9-08eef07bdefb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1b268763c624098a3824e1b47a0cbb0'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:55.573162 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-01fc6947-6d52-47b3-b6a9-08eef07bdefb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1b268763c624098a3824e1b47a0cbb0'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:55.615562 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-01fc6947-6d52-47b3-b6a9-08eef07bdefb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1b268763c624098a3824e1b47a0cbb0'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:55.619270 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-01fc6947-6d52-47b3-b6a9-08eef07bdefb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1b268763c624098a3824e1b47a0cbb0'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:55.762355 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1039/2091] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:53:54 2022] DELETE /identity/v3/users/e1b268763c624098a3824e1b47a0cbb0 => generated 0 bytes in 826 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:56.469992 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e7fae-1f35-4a05-a8a4-9f277c0439c1 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:56.473634 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-586e7fae-1f35-4a05-a8a4-9f277c0439c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:56.523331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e7fae-1f35-4a05-a8a4-9f277c0439c1 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:56.637106 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e38bd-f173-4f72-84ea-8188580a1ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:56.638040 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e38bd-f173-4f72-84ea-8188580a1ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:56.638499 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e38bd-f173-4f72-84ea-8188580a1ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:56.639295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e38bd-f173-4f72-84ea-8188580a1ca9 admin admin] PATH_INFO: `/v3/projects/9ce3bd673a6a488a95a3defe50f6d541` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:56.641138 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e38bd-f173-4f72-84ea-8188580a1ca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ce3bd673a6a488a95a3defe50f6d541)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:56.644211 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e38bd-f173-4f72-84ea-8188580a1ca9 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:56.672990 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:56.799433 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-2c0e38bd-f173-4f72-84ea-8188580a1ca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ce3bd673a6a488a95a3defe50f6d541', 'request_id': 'req-2c0e38bd-f173-4f72-84ea-8188580a1ca9'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:56.931003 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-2c0e38bd-f173-4f72-84ea-8188580a1ca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ce3bd673a6a488a95a3defe50f6d541', 'request_id': 'req-2c0e38bd-f173-4f72-84ea-8188580a1ca9'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:56.946768 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-2c0e38bd-f173-4f72-84ea-8188580a1ca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ce3bd673a6a488a95a3defe50f6d541', 'request_id': 'req-2c0e38bd-f173-4f72-84ea-8188580a1ca9'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:56.971283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-2c0e38bd-f173-4f72-84ea-8188580a1ca9 admin admin] The token cache is being invalidate because project 9ce3bd673a6a488a95a3defe50f6d541 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:53:56.972659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-2c0e38bd-f173-4f72-84ea-8188580a1ca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:56.981668 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-2c0e38bd-f173-4f72-84ea-8188580a1ca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:56.984280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1053/2092] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:53:56 2022] DELETE /identity/v3/projects/9ce3bd673a6a488a95a3defe50f6d541 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:56.992362 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06069556-8f3b-42c0-a889-4c2f0493d845 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:56.993109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06069556-8f3b-42c0-a889-4c2f0493d845 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:56.993324 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06069556-8f3b-42c0-a889-4c2f0493d845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:57.095289 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-06069556-8f3b-42c0-a889-4c2f0493d845 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:57.098717 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-06069556-8f3b-42c0-a889-4c2f0493d845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:57.261466 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1040/2093] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:57.275095 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579e9fc-d0bb-474e-b30c-f534c458b5c4 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:57.275966 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579e9fc-d0bb-474e-b30c-f534c458b5c4 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:57.276619 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579e9fc-d0bb-474e-b30c-f534c458b5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:57.346251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-8579e9fc-d0bb-474e-b30c-f534c458b5c4 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:57.350142 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-8579e9fc-d0bb-474e-b30c-f534c458b5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:57.395792 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1054/2094] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:57.400356 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06069556-8f3b-42c0-a889-4c2f0493d845 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:57.418854 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06069556-8f3b-42c0-a889-4c2f0493d845 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:57.441886 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd27ebd-800d-4084-914d-af1d29ebf032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:57.442522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd27ebd-800d-4084-914d-af1d29ebf032 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:57.442835 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd27ebd-800d-4084-914d-af1d29ebf032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:57.443095 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd27ebd-800d-4084-914d-af1d29ebf032 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:57.443763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd27ebd-800d-4084-914d-af1d29ebf032 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:53:57.444065 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd27ebd-800d-4084-914d-af1d29ebf032 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:57.445728 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd27ebd-800d-4084-914d-af1d29ebf032 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:57.480785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1041/2095] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:53:57.487909 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579e9fc-d0bb-474e-b30c-f534c458b5c4 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:57.507683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579e9fc-d0bb-474e-b30c-f534c458b5c4 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:57.514016 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fe827-8f73-4911-ba92-c8cc79930195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:57.516881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fe827-8f73-4911-ba92-c8cc79930195 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:57.517171 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fe827-8f73-4911-ba92-c8cc79930195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:57.517448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fe827-8f73-4911-ba92-c8cc79930195 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:57.518117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fe827-8f73-4911-ba92-c8cc79930195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:57.520669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fe827-8f73-4911-ba92-c8cc79930195 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:57.608050 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd27ebd-800d-4084-914d-af1d29ebf032 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:57.610680 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-edd27ebd-800d-4084-914d-af1d29ebf032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:57.627532 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd27ebd-800d-4084-914d-af1d29ebf032 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:57.635139 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a52ce0-4819-40da-b9c9-22fba1f0dbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:57.635803 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a52ce0-4819-40da-b9c9-22fba1f0dbc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:57.636999 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a52ce0-4819-40da-b9c9-22fba1f0dbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:57.636999 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a52ce0-4819-40da-b9c9-22fba1f0dbc7 admin admin] PATH_INFO: `/v3/users/f950627ca8544c2f80f2a4f12a4c0bdb` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:57.638013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a52ce0-4819-40da-b9c9-22fba1f0dbc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f950627ca8544c2f80f2a4f12a4c0bdb)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:57.640112 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a52ce0-4819-40da-b9c9-22fba1f0dbc7 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:57.682618 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1055/2096] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:53:57 2022] POST /identity/v3/projects => generated 407 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:53:57.683956 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fe827-8f73-4911-ba92-c8cc79930195 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:57.798977 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fe827-8f73-4911-ba92-c8cc79930195 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:57.805489 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce054a1e-1794-4681-a1a2-f1daa7d32375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:57.806367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce054a1e-1794-4681-a1a2-f1daa7d32375 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:57.806826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce054a1e-1794-4681-a1a2-f1daa7d32375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:57.807251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce054a1e-1794-4681-a1a2-f1daa7d32375 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:57.811983 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce054a1e-1794-4681-a1a2-f1daa7d32375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:57.816018 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce054a1e-1794-4681-a1a2-f1daa7d32375 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:57.886435 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-42a52ce0-4819-40da-b9c9-22fba1f0dbc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f950627ca8544c2f80f2a4f12a4c0bdb'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:57.970385 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1056/2097] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:53:57 2022] POST /identity/v3/users => generated 304 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:57.975013 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce054a1e-1794-4681-a1a2-f1daa7d32375 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:57.987956 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-42a52ce0-4819-40da-b9c9-22fba1f0dbc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f950627ca8544c2f80f2a4f12a4c0bdb'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:57.988593 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-42a52ce0-4819-40da-b9c9-22fba1f0dbc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f950627ca8544c2f80f2a4f12a4c0bdb'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:58.000129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce054a1e-1794-4681-a1a2-f1daa7d32375 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:58.008069 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348b112-1659-486e-be28-6fca9c997852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:58.009107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348b112-1659-486e-be28-6fca9c997852 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:58.009818 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348b112-1659-486e-be28-6fca9c997852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:58.010265 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348b112-1659-486e-be28-6fca9c997852 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:58.011256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a348b112-1659-486e-be28-6fca9c997852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:58.013046 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a348b112-1659-486e-be28-6fca9c997852 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:58.017719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-42a52ce0-4819-40da-b9c9-22fba1f0dbc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f950627ca8544c2f80f2a4f12a4c0bdb'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:58.023458 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-42a52ce0-4819-40da-b9c9-22fba1f0dbc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f950627ca8544c2f80f2a4f12a4c0bdb'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:58.028152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1057/2098] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:53:58.033121 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348b112-1659-486e-be28-6fca9c997852 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:58.035648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-a348b112-1659-486e-be28-6fca9c997852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:58.054979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348b112-1659-486e-be28-6fca9c997852 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:58.062967 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff54250-bac4-4326-aa8b-1971e85d3f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:58.063937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff54250-bac4-4326-aa8b-1971e85d3f0b admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:58.064368 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff54250-bac4-4326-aa8b-1971e85d3f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:58.064791 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff54250-bac4-4326-aa8b-1971e85d3f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:58.066509 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff54250-bac4-4326-aa8b-1971e85d3f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:58.069593 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff54250-bac4-4326-aa8b-1971e85d3f0b admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:58.080071 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:58.101447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1042/2099] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:53:57 2022] DELETE /identity/v3/users/f950627ca8544c2f80f2a4f12a4c0bdb => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:58.107694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e5a2f-cf00-4560-9046-e1015713c9a7 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:58.109595 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e5a2f-cf00-4560-9046-e1015713c9a7 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:58.111730 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e5a2f-cf00-4560-9046-e1015713c9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:58.123796 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1058/2100] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:53:58.129112 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff54250-bac4-4326-aa8b-1971e85d3f0b admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:58.144710 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff54250-bac4-4326-aa8b-1971e85d3f0b admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:58.150555 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-eb3e5a2f-cf00-4560-9046-e1015713c9a7 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:58.152067 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-eb3e5a2f-cf00-4560-9046-e1015713c9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:58.274805 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1043/2101] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:58.275071 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272aeca-031c-4c0c-ba5b-6db14c65755b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:58.275777 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272aeca-031c-4c0c-ba5b-6db14c65755b admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:58.276052 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272aeca-031c-4c0c-ba5b-6db14c65755b admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:58.276478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272aeca-031c-4c0c-ba5b-6db14c65755b admin admin] PATH_INFO: `/v3/projects/7cafcc6cb5c749e297eea217a0e53f94/users/9f32d71a19e745f19e70c60c89e04db7/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:58.293177 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e5a2f-cf00-4560-9046-e1015713c9a7 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:58.295040 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-eb3e5a2f-cf00-4560-9046-e1015713c9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:58.312386 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1272aeca-031c-4c0c-ba5b-6db14c65755b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cafcc6cb5c749e297eea217a0e53f94, user_id=9f32d71a19e745f19e70c60c89e04db7, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:58.319335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1272aeca-031c-4c0c-ba5b-6db14c65755b admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:58.388207 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e5a2f-cf00-4560-9046-e1015713c9a7 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:58.399493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e338362-bfa2-4599-814f-742c66d93492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:58.404604 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e338362-bfa2-4599-814f-742c66d93492 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:58.404604 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e338362-bfa2-4599-814f-742c66d93492 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:58.404604 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e338362-bfa2-4599-814f-742c66d93492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:58.425314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e338362-bfa2-4599-814f-742c66d93492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:58.427146 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e338362-bfa2-4599-814f-742c66d93492 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:58.467577 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1044/2102] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:53:58.477008 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1059/2103] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:58 2022] PUT /identity/v3/projects/7cafcc6cb5c749e297eea217a0e53f94/users/9f32d71a19e745f19e70c60c89e04db7/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:58.499873 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5e506-0f55-4993-9d85-17a9c79b366a None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:58.500806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5e506-0f55-4993-9d85-17a9c79b366a None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:58.500931 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5e506-0f55-4993-9d85-17a9c79b366a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:58.572037 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-0fd5e506-0f55-4993-9d85-17a9c79b366a None None] MFA Rules not processed for user `9f32d71a19e745f19e70c60c89e04db7`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:58.576406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-0fd5e506-0f55-4993-9d85-17a9c79b366a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:58.652100 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:58.714490 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1045/2104] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:53:58 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:58.721254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272aeca-031c-4c0c-ba5b-6db14c65755b admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:58.820337 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272aeca-031c-4c0c-ba5b-6db14c65755b admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:58.837036 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a2687-bad7-49a5-8ade-e43e5b55ed3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:58.838789 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a2687-bad7-49a5-8ade-e43e5b55ed3a admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:58.838789 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a2687-bad7-49a5-8ade-e43e5b55ed3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:58.838789 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a2687-bad7-49a5-8ade-e43e5b55ed3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:58.841132 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a2687-bad7-49a5-8ade-e43e5b55ed3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:58.843686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a2687-bad7-49a5-8ade-e43e5b55ed3a admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:58.973946 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1060/2105] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:53:58 2022] POST /identity/v3/projects => generated 409 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:58.982813 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5e506-0f55-4993-9d85-17a9c79b366a None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:59.007730 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5e506-0f55-4993-9d85-17a9c79b366a None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:59.195992 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ef398-8eda-47ba-a511-b0ed44289960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:59.196903 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ef398-8eda-47ba-a511-b0ed44289960 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:59.197252 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ef398-8eda-47ba-a511-b0ed44289960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:59.198735 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ef398-8eda-47ba-a511-b0ed44289960 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:59.199684 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ef398-8eda-47ba-a511-b0ed44289960 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:59.201677 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ef398-8eda-47ba-a511-b0ed44289960 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:59.239454 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a2687-bad7-49a5-8ade-e43e5b55ed3a admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:59.241670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-153a2687-bad7-49a5-8ade-e43e5b55ed3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:59.265077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a2687-bad7-49a5-8ade-e43e5b55ed3a admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:59.271368 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66d585-c53c-4bf8-b1e4-0314f3a224f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:59.273165 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66d585-c53c-4bf8-b1e4-0314f3a224f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:59.273615 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66d585-c53c-4bf8-b1e4-0314f3a224f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:59.274062 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66d585-c53c-4bf8-b1e4-0314f3a224f1 admin admin] PATH_INFO: `/v3/users/a176a19c3d924de5bd60d62c6c36839d` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:59.276174 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e66d585-c53c-4bf8-b1e4-0314f3a224f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a176a19c3d924de5bd60d62c6c36839d)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:59.278891 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e66d585-c53c-4bf8-b1e4-0314f3a224f1 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:59.384446 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1046/2106] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:53:58 2022] POST /identity/v3/users => generated 311 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:53:59.390764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ef398-8eda-47ba-a511-b0ed44289960 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:59.410540 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ef398-8eda-47ba-a511-b0ed44289960 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:59.419059 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ef9ff-7fb2-4d2a-809e-db88fd4d0082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:59.420116 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ef9ff-7fb2-4d2a-809e-db88fd4d0082 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:59.420603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ef9ff-7fb2-4d2a-809e-db88fd4d0082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:59.421069 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ef9ff-7fb2-4d2a-809e-db88fd4d0082 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:59.422264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ef9ff-7fb2-4d2a-809e-db88fd4d0082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:59.426168 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ef9ff-7fb2-4d2a-809e-db88fd4d0082 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:59.457129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1047/2107] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:53:59.459319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ef9ff-7fb2-4d2a-809e-db88fd4d0082 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:59.497676 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ef9ff-7fb2-4d2a-809e-db88fd4d0082 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:59.517447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e879b-865f-4ded-95de-58ad004c9d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:59.519500 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e879b-865f-4ded-95de-58ad004c9d1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:59.528487 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e879b-865f-4ded-95de-58ad004c9d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:59.528487 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e879b-865f-4ded-95de-58ad004c9d1b admin admin] PATH_INFO: `/v3/projects/b69b9d9c02794402bb1c0aacf686c87e/users/c481bedc075342aaae758e0487726ef1/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:59.558101 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e879b-865f-4ded-95de-58ad004c9d1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b69b9d9c02794402bb1c0aacf686c87e, user_id=c481bedc075342aaae758e0487726ef1, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:59.565030 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e879b-865f-4ded-95de-58ad004c9d1b admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:59.614786 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3e66d585-c53c-4bf8-b1e4-0314f3a224f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a176a19c3d924de5bd60d62c6c36839d'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:59.685619 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1048/2108] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:53:59 2022] PUT /identity/v3/projects/b69b9d9c02794402bb1c0aacf686c87e/users/c481bedc075342aaae758e0487726ef1/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:53:59.693268 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e879b-865f-4ded-95de-58ad004c9d1b admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:59.712800 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3e66d585-c53c-4bf8-b1e4-0314f3a224f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a176a19c3d924de5bd60d62c6c36839d'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:59.716755 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3e66d585-c53c-4bf8-b1e4-0314f3a224f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a176a19c3d924de5bd60d62c6c36839d'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:59.754279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3e66d585-c53c-4bf8-b1e4-0314f3a224f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a176a19c3d924de5bd60d62c6c36839d'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:59.805724 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3e66d585-c53c-4bf8-b1e4-0314f3a224f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a176a19c3d924de5bd60d62c6c36839d'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:53:59.820886 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1061/2109] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:53:59 2022] DELETE /identity/v3/users/a176a19c3d924de5bd60d62c6c36839d => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:53:59.822937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66d585-c53c-4bf8-b1e4-0314f3a224f1 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:59.829083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-3e66d585-c53c-4bf8-b1e4-0314f3a224f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:59.902006 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e879b-865f-4ded-95de-58ad004c9d1b admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:59.936047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66d585-c53c-4bf8-b1e4-0314f3a224f1 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:59.952796 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc731db9-9560-4cb8-a128-a7a9c36de4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:59.954884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc731db9-9560-4cb8-a128-a7a9c36de4fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:59.955647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc731db9-9560-4cb8-a128-a7a9c36de4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:59.956354 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc731db9-9560-4cb8-a128-a7a9c36de4fa admin admin] PATH_INFO: `/v3/projects/f05babb5bf2d4497b245ebb198673e88` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:59.961331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc731db9-9560-4cb8-a128-a7a9c36de4fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=f05babb5bf2d4497b245ebb198673e88)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:59.967324 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc731db9-9560-4cb8-a128-a7a9c36de4fa admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:00.049018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bfe876-f67a-405e-b0fc-fde41a26f177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:00.050885 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfe876-f67a-405e-b0fc-fde41a26f177 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:00.051959 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfe876-f67a-405e-b0fc-fde41a26f177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:00.052492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfe876-f67a-405e-b0fc-fde41a26f177 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:00.053775 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bfe876-f67a-405e-b0fc-fde41a26f177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:00.056862 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bfe876-f67a-405e-b0fc-fde41a26f177 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:00.085437 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1049/2110] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:54:00.088005 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b403c93-f8cf-48d8-b136-d1c6142788b3 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:00.088530 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b403c93-f8cf-48d8-b136-d1c6142788b3 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:00.089031 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b403c93-f8cf-48d8-b136-d1c6142788b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:00.127341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-9b403c93-f8cf-48d8-b136-d1c6142788b3 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:00.130202 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-9b403c93-f8cf-48d8-b136-d1c6142788b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:00.151513 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1050/2111] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:00.155002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b403c93-f8cf-48d8-b136-d1c6142788b3 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:00.157766 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-9b403c93-f8cf-48d8-b136-d1c6142788b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:00.178205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-fc731db9-9560-4cb8-a128-a7a9c36de4fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f05babb5bf2d4497b245ebb198673e88', 'request_id': 'req-fc731db9-9560-4cb8-a128-a7a9c36de4fa'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:00.192346 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b403c93-f8cf-48d8-b136-d1c6142788b3 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:00.233138 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-fc731db9-9560-4cb8-a128-a7a9c36de4fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f05babb5bf2d4497b245ebb198673e88', 'request_id': 'req-fc731db9-9560-4cb8-a128-a7a9c36de4fa'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:00.250541 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-fc731db9-9560-4cb8-a128-a7a9c36de4fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f05babb5bf2d4497b245ebb198673e88', 'request_id': 'req-fc731db9-9560-4cb8-a128-a7a9c36de4fa'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:00.274212 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-fc731db9-9560-4cb8-a128-a7a9c36de4fa admin admin] The token cache is being invalidate because project f05babb5bf2d4497b245ebb198673e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:54:00.275154 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-fc731db9-9560-4cb8-a128-a7a9c36de4fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:00.276698 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-fc731db9-9560-4cb8-a128-a7a9c36de4fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:00.279805 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1062/2112] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:53:59 2022] DELETE /identity/v3/projects/f05babb5bf2d4497b245ebb198673e88 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:54:00.281371 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc731db9-9560-4cb8-a128-a7a9c36de4fa admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:00.291647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-fc731db9-9560-4cb8-a128-a7a9c36de4fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:00.376418 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3c224-a4df-4d89-a354-b53f4f013ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:00.377142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3c224-a4df-4d89-a354-b53f4f013ce3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:00.377639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3c224-a4df-4d89-a354-b53f4f013ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:00.378882 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3c224-a4df-4d89-a354-b53f4f013ce3 admin admin] PATH_INFO: `/v3/projects/b69b9d9c02794402bb1c0aacf686c87e/users/c481bedc075342aaae758e0487726ef1/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:00.402807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3c224-a4df-4d89-a354-b53f4f013ce3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b69b9d9c02794402bb1c0aacf686c87e, user_id=c481bedc075342aaae758e0487726ef1, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:00.411013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3c224-a4df-4d89-a354-b53f4f013ce3 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:00.440586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1051/2113] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:54:00 2022] PUT /identity/v3/projects/b69b9d9c02794402bb1c0aacf686c87e/users/c481bedc075342aaae758e0487726ef1/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:54:00.443766 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad189a25-9632-4fc0-b8e7-873a69b5ab9a None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:00.444365 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad189a25-9632-4fc0-b8e7-873a69b5ab9a None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:00.444876 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad189a25-9632-4fc0-b8e7-873a69b5ab9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:00.523385 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc731db9-9560-4cb8-a128-a7a9c36de4fa admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:00.553681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-ad189a25-9632-4fc0-b8e7-873a69b5ab9a None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:00.553681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ad189a25-9632-4fc0-b8e7-873a69b5ab9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:00.657146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce3853-9cae-47b0-9f63-5f29d89f9af6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:00.657966 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce3853-9cae-47b0-9f63-5f29d89f9af6 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:00.658413 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce3853-9cae-47b0-9f63-5f29d89f9af6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:00.658902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce3853-9cae-47b0-9f63-5f29d89f9af6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:00.662439 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-71ce3853-9cae-47b0-9f63-5f29d89f9af6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:00.672842 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1052/2114] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:00.675204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e727d1-ce9e-4b12-a7a4-a4844125c82b None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:00.675870 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e727d1-ce9e-4b12-a7a4-a4844125c82b None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:00.676329 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e727d1-ce9e-4b12-a7a4-a4844125c82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:00.687155 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce3853-9cae-47b0-9f63-5f29d89f9af6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:00.689894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce3853-9cae-47b0-9f63-5f29d89f9af6 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:00.710118 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1063/2115] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:54:00.712593 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4fe3e-d5eb-4b8b-8d3d-24a372c850d1 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:00.718419 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4fe3e-d5eb-4b8b-8d3d-24a372c850d1 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:00.718419 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4fe3e-d5eb-4b8b-8d3d-24a372c850d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:00.729160 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-25e727d1-ce9e-4b12-a7a4-a4844125c82b None None] MFA Rules not processed for user `c481bedc075342aaae758e0487726ef1`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:00.731093 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-25e727d1-ce9e-4b12-a7a4-a4844125c82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:00.751400 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-84b4fe3e-d5eb-4b8b-8d3d-24a372c850d1 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:00.755207 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-84b4fe3e-d5eb-4b8b-8d3d-24a372c850d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:00.766156 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1064/2116] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:00.772684 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4fe3e-d5eb-4b8b-8d3d-24a372c850d1 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:00.797631 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4fe3e-d5eb-4b8b-8d3d-24a372c850d1 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:00.807509 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0ddd5-7e55-4537-adfa-beea0e4c0563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:00.809107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0ddd5-7e55-4537-adfa-beea0e4c0563 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:00.809721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0ddd5-7e55-4537-adfa-beea0e4c0563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:00.810340 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0ddd5-7e55-4537-adfa-beea0e4c0563 admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:00.812435 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0ddd5-7e55-4537-adfa-beea0e4c0563 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:54:00.813142 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0ddd5-7e55-4537-adfa-beea0e4c0563 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:00.816402 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0ddd5-7e55-4537-adfa-beea0e4c0563 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:00.839310 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1065/2117] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:54:00.841152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0ddd5-7e55-4537-adfa-beea0e4c0563 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:00.879778 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0ddd5-7e55-4537-adfa-beea0e4c0563 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:00.886066 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:00.895699 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06524a6d-5e3d-4f8e-b683-26a2c14b4492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:00.898329 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06524a6d-5e3d-4f8e-b683-26a2c14b4492 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:00.898720 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06524a6d-5e3d-4f8e-b683-26a2c14b4492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:00.899065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06524a6d-5e3d-4f8e-b683-26a2c14b4492 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:00.899816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06524a6d-5e3d-4f8e-b683-26a2c14b4492 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:00.903192 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06524a6d-5e3d-4f8e-b683-26a2c14b4492 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:00.990706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1053/2118] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:00 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:01.010961 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e727d1-ce9e-4b12-a7a4-a4844125c82b None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:01.012780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-25e727d1-ce9e-4b12-a7a4-a4844125c82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:01.087282 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1066/2119] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:54:00 2022] POST /identity/v3/projects => generated 375 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:54:01.093143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06524a6d-5e3d-4f8e-b683-26a2c14b4492 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:01.117049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e727d1-ce9e-4b12-a7a4-a4844125c82b None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:01.199452 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2753e9-e666-471a-87d4-614052e3a3ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:01.200758 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2753e9-e666-471a-87d4-614052e3a3ff service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:01.201240 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2753e9-e666-471a-87d4-614052e3a3ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:01.201716 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2753e9-e666-471a-87d4-614052e3a3ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:01.282366 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2753e9-e666-471a-87d4-614052e3a3ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:01.285148 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2753e9-e666-471a-87d4-614052e3a3ff service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:01.297565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1054/2120] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:01 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:54:01.422802 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06524a6d-5e3d-4f8e-b683-26a2c14b4492 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:01.444504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93c52b-d1fe-44f7-bfd7-63599dbdb08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:01.464342 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93c52b-d1fe-44f7-bfd7-63599dbdb08b admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:01.464342 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93c52b-d1fe-44f7-bfd7-63599dbdb08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:01.464342 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93c52b-d1fe-44f7-bfd7-63599dbdb08b admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:01.467629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93c52b-d1fe-44f7-bfd7-63599dbdb08b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:01.473277 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93c52b-d1fe-44f7-bfd7-63599dbdb08b admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:01.670002 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1067/2121] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:54:01 2022] POST /identity/v3/users => generated 288 bytes in 582 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:54:01.675740 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2753e9-e666-471a-87d4-614052e3a3ff service cinder] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:01.713527 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2753e9-e666-471a-87d4-614052e3a3ff service cinder] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:01.720932 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:01.738300 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac61b2f-1294-4f0f-aeb1-a66f1352e73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:01.739452 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac61b2f-1294-4f0f-aeb1-a66f1352e73b admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:01.739935 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac61b2f-1294-4f0f-aeb1-a66f1352e73b admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:01.740424 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac61b2f-1294-4f0f-aeb1-a66f1352e73b admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:01.741646 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac61b2f-1294-4f0f-aeb1-a66f1352e73b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:01.744649 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac61b2f-1294-4f0f-aeb1-a66f1352e73b admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:01.771324 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1055/2122] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:54:01.776364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93c52b-d1fe-44f7-bfd7-63599dbdb08b admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:01.796727 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93c52b-d1fe-44f7-bfd7-63599dbdb08b admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:01.816914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011d733-bcfe-4498-ab74-2d48362af628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:01.818065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011d733-bcfe-4498-ab74-2d48362af628 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:01.820080 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011d733-bcfe-4498-ab74-2d48362af628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:01.820557 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011d733-bcfe-4498-ab74-2d48362af628 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:01.822211 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011d733-bcfe-4498-ab74-2d48362af628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:01.843415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011d733-bcfe-4498-ab74-2d48362af628 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:01.864311 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:01.884453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1068/2123] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:54:01.886366 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac61b2f-1294-4f0f-aeb1-a66f1352e73b admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:01.908389 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac61b2f-1294-4f0f-aeb1-a66f1352e73b admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:01.918271 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd8b0f-f21d-4419-ad90-4a49717e59f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:01.920897 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd8b0f-f21d-4419-ad90-4a49717e59f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:01.920897 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd8b0f-f21d-4419-ad90-4a49717e59f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:01.920897 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd8b0f-f21d-4419-ad90-4a49717e59f9 admin admin] PATH_INFO: `/v3/projects/e475c58bc22940dea4bc85a183ac7053/users/eef585b05da2444cbe8b715a55a5bcc6/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:01.962208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd8b0f-f21d-4419-ad90-4a49717e59f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e475c58bc22940dea4bc85a183ac7053, user_id=eef585b05da2444cbe8b715a55a5bcc6, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:01.965777 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd8b0f-f21d-4419-ad90-4a49717e59f9 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:02.035761 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011d733-bcfe-4498-ab74-2d48362af628 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:02.090029 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1056/2124] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:54:01 2022] PUT /identity/v3/projects/e475c58bc22940dea4bc85a183ac7053/users/eef585b05da2444cbe8b715a55a5bcc6/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:54:02.092682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011d733-bcfe-4498-ab74-2d48362af628 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:02.103536 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451634a-283a-4834-85cb-6d76999458c6 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:02.105142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451634a-283a-4834-85cb-6d76999458c6 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:02.106701 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451634a-283a-4834-85cb-6d76999458c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:02.174286 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-8451634a-283a-4834-85cb-6d76999458c6 None None] MFA Rules not processed for user `eef585b05da2444cbe8b715a55a5bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:02.174286 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-8451634a-283a-4834-85cb-6d76999458c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:02.231049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7b893-1442-460c-a1f2-5a61001d1545 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:02.231049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7b893-1442-460c-a1f2-5a61001d1545 service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:02.231049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7b893-1442-460c-a1f2-5a61001d1545 service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:02.231049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7b893-1442-460c-a1f2-5a61001d1545 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:02.231686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-fcd7b893-1442-460c-a1f2-5a61001d1545 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:02.345840 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1057/2125] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:02.345840 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451634a-283a-4834-85cb-6d76999458c6 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:02.355128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-8451634a-283a-4834-85cb-6d76999458c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:02.429056 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd7b893-1442-460c-a1f2-5a61001d1545 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:02.438968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd7b893-1442-460c-a1f2-5a61001d1545 service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:02.472184 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1069/2126] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:02 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:54:02.473217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7b893-1442-460c-a1f2-5a61001d1545 service cinder] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:02.474755 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-fcd7b893-1442-460c-a1f2-5a61001d1545 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:02.493864 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7b893-1442-460c-a1f2-5a61001d1545 service cinder] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:02.500676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f4da9-adde-4904-8c82-d7532bfe044a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:02.501543 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f4da9-adde-4904-8c82-d7532bfe044a service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:02.501896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f4da9-adde-4904-8c82-d7532bfe044a service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:02.502256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f4da9-adde-4904-8c82-d7532bfe044a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:02.542932 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f4da9-adde-4904-8c82-d7532bfe044a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:02.547364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f4da9-adde-4904-8c82-d7532bfe044a service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:02.578183 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1070/2127] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:54:02.616325 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451634a-283a-4834-85cb-6d76999458c6 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:02.625716 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694af16-6b8d-4500-b1ff-16c165067184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:02.640579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694af16-6b8d-4500-b1ff-16c165067184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:02.640579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694af16-6b8d-4500-b1ff-16c165067184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:02.640579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694af16-6b8d-4500-b1ff-16c165067184 admin admin] PATH_INFO: `/v3/projects/ec2f75f8c92f4eca97ce6c701ef99d99` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:02.640892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3694af16-6b8d-4500-b1ff-16c165067184 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec2f75f8c92f4eca97ce6c701ef99d99)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:02.655592 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3694af16-6b8d-4500-b1ff-16c165067184 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:02.752841 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080c246-1abd-446e-8935-b7bb7e9107c2 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:02.753843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080c246-1abd-446e-8935-b7bb7e9107c2 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:02.754288 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080c246-1abd-446e-8935-b7bb7e9107c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:02.755750 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:02.817160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-4080c246-1abd-446e-8935-b7bb7e9107c2 None None] MFA Rules not processed for user `d6d78cbc233a498db3a9268a4b93d737`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:02.819147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-4080c246-1abd-446e-8935-b7bb7e9107c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:02.867346 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-3694af16-6b8d-4500-b1ff-16c165067184 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec2f75f8c92f4eca97ce6c701ef99d99', 'request_id': 'req-3694af16-6b8d-4500-b1ff-16c165067184'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:02.912919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-3694af16-6b8d-4500-b1ff-16c165067184 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec2f75f8c92f4eca97ce6c701ef99d99', 'request_id': 'req-3694af16-6b8d-4500-b1ff-16c165067184'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:02.931271 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1071/2128] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:54:02.955473 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080c246-1abd-446e-8935-b7bb7e9107c2 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:02.958327 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-4080c246-1abd-446e-8935-b7bb7e9107c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:02.966253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-3694af16-6b8d-4500-b1ff-16c165067184 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec2f75f8c92f4eca97ce6c701ef99d99', 'request_id': 'req-3694af16-6b8d-4500-b1ff-16c165067184'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:03.000679 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:03.056317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-3694af16-6b8d-4500-b1ff-16c165067184 admin admin] The token cache is being invalidate because project ec2f75f8c92f4eca97ce6c701ef99d99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:54:03.057339 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-3694af16-6b8d-4500-b1ff-16c165067184 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:03.060025 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-3694af16-6b8d-4500-b1ff-16c165067184 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:03.063847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1058/2129] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:54:02 2022] DELETE /identity/v3/projects/ec2f75f8c92f4eca97ce6c701ef99d99 => generated 0 bytes in 720 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:54:03.069609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694af16-6b8d-4500-b1ff-16c165067184 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:03.073320 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-3694af16-6b8d-4500-b1ff-16c165067184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:03.144411 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080c246-1abd-446e-8935-b7bb7e9107c2 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:03.179713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694af16-6b8d-4500-b1ff-16c165067184 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:03.185358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa6ebd-d660-4d50-816e-c7248e8b3d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:03.186072 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa6ebd-d660-4d50-816e-c7248e8b3d77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:03.186353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa6ebd-d660-4d50-816e-c7248e8b3d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:03.186661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa6ebd-d660-4d50-816e-c7248e8b3d77 admin admin] PATH_INFO: `/v3/users/699a2d5b488f4271ac38af00581b8f9e` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:03.188028 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa6ebd-d660-4d50-816e-c7248e8b3d77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=699a2d5b488f4271ac38af00581b8f9e)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:03.193016 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa6ebd-d660-4d50-816e-c7248e8b3d77 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:03.253751 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c2cec-f53d-49bf-b5c8-ccb8358d4d43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:03.254985 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c2cec-f53d-49bf-b5c8-ccb8358d4d43 service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:03.255479 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c2cec-f53d-49bf-b5c8-ccb8358d4d43 service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:03.255991 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c2cec-f53d-49bf-b5c8-ccb8358d4d43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:03.263019 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-665c2cec-f53d-49bf-b5c8-ccb8358d4d43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:03.438919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c2cec-f53d-49bf-b5c8-ccb8358d4d43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:03.444353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c2cec-f53d-49bf-b5c8-ccb8358d4d43 service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:03.468064 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1072/2130] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:54:03.499795 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-76fa6ebd-d660-4d50-816e-c7248e8b3d77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '699a2d5b488f4271ac38af00581b8f9e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:03.616428 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-76fa6ebd-d660-4d50-816e-c7248e8b3d77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '699a2d5b488f4271ac38af00581b8f9e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:03.618940 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-76fa6ebd-d660-4d50-816e-c7248e8b3d77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '699a2d5b488f4271ac38af00581b8f9e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:03.647279 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-76fa6ebd-d660-4d50-816e-c7248e8b3d77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '699a2d5b488f4271ac38af00581b8f9e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:03.650434 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-76fa6ebd-d660-4d50-816e-c7248e8b3d77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '699a2d5b488f4271ac38af00581b8f9e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:03.715216 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1059/2131] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:54:03 2022] DELETE /identity/v3/users/699a2d5b488f4271ac38af00581b8f9e => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:54:03.935878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:04.503706 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c2cec-f53d-49bf-b5c8-ccb8358d4d43 service nova] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:04.532809 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c2cec-f53d-49bf-b5c8-ccb8358d4d43 service nova] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:04.655638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73b954-28ef-4a7a-a10b-4f10812f9fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:04.656663 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73b954-28ef-4a7a-a10b-4f10812f9fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:04.656663 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73b954-28ef-4a7a-a10b-4f10812f9fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:04.656954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73b954-28ef-4a7a-a10b-4f10812f9fee admin admin] PATH_INFO: `/v3/projects/5688be2745b448869bef7035fc950da7` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:04.658306 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73b954-28ef-4a7a-a10b-4f10812f9fee admin admin] RBAC: Authorizing `identity:delete_project(project_id=5688be2745b448869bef7035fc950da7)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:04.661718 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73b954-28ef-4a7a-a10b-4f10812f9fee admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:04.824190 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-9c73b954-28ef-4a7a-a10b-4f10812f9fee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5688be2745b448869bef7035fc950da7', 'request_id': 'req-9c73b954-28ef-4a7a-a10b-4f10812f9fee'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:04.947376 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-9c73b954-28ef-4a7a-a10b-4f10812f9fee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5688be2745b448869bef7035fc950da7', 'request_id': 'req-9c73b954-28ef-4a7a-a10b-4f10812f9fee'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:04.964053 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-9c73b954-28ef-4a7a-a10b-4f10812f9fee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5688be2745b448869bef7035fc950da7', 'request_id': 'req-9c73b954-28ef-4a7a-a10b-4f10812f9fee'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:04.986930 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-9c73b954-28ef-4a7a-a10b-4f10812f9fee admin admin] The token cache is being invalidate because project 5688be2745b448869bef7035fc950da7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:54:04.987983 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-9c73b954-28ef-4a7a-a10b-4f10812f9fee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:04.991165 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-9c73b954-28ef-4a7a-a10b-4f10812f9fee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:04.993270 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1073/2132] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:54:04 2022] DELETE /identity/v3/projects/5688be2745b448869bef7035fc950da7 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:54:04.997360 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa6ebd-d660-4d50-816e-c7248e8b3d77 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:04.999892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-76fa6ebd-d660-4d50-816e-c7248e8b3d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:05.016463 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa6ebd-d660-4d50-816e-c7248e8b3d77 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:05.086454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58381702-a114-4d46-9bcd-d1773a94a47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:05.087133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58381702-a114-4d46-9bcd-d1773a94a47f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:05.087412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58381702-a114-4d46-9bcd-d1773a94a47f admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:05.087685 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58381702-a114-4d46-9bcd-d1773a94a47f admin admin] PATH_INFO: `/v3/users/06b1df855c10450f973fbf4c4c08a520` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:05.088900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58381702-a114-4d46-9bcd-d1773a94a47f admin admin] RBAC: Authorizing `identity:delete_user(user_id=06b1df855c10450f973fbf4c4c08a520)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:05.091046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58381702-a114-4d46-9bcd-d1773a94a47f admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:05.278667 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:05.467225 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-58381702-a114-4d46-9bcd-d1773a94a47f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06b1df855c10450f973fbf4c4c08a520'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:05.588165 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-58381702-a114-4d46-9bcd-d1773a94a47f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06b1df855c10450f973fbf4c4c08a520'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:05.591184 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-58381702-a114-4d46-9bcd-d1773a94a47f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06b1df855c10450f973fbf4c4c08a520'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:05.615980 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-58381702-a114-4d46-9bcd-d1773a94a47f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06b1df855c10450f973fbf4c4c08a520'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:05.617109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-58381702-a114-4d46-9bcd-d1773a94a47f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06b1df855c10450f973fbf4c4c08a520'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:05.643415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1060/2133] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:54:04 2022] DELETE /identity/v3/users/06b1df855c10450f973fbf4c4c08a520 => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:54:05.794266 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:06.415154 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73b954-28ef-4a7a-a10b-4f10812f9fee admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:06.419578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-9c73b954-28ef-4a7a-a10b-4f10812f9fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:06.466626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73b954-28ef-4a7a-a10b-4f10812f9fee admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:06.641189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e6c04-1405-4f3f-87d5-84643ae3f228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:06.643387 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e6c04-1405-4f3f-87d5-84643ae3f228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:06.643387 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e6c04-1405-4f3f-87d5-84643ae3f228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:06.647951 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e6c04-1405-4f3f-87d5-84643ae3f228 admin admin] PATH_INFO: `/v3/projects/2b586d4b3b5a4739b6d81e7ed0e62fc6` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:06.652000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e6c04-1405-4f3f-87d5-84643ae3f228 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b586d4b3b5a4739b6d81e7ed0e62fc6)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:06.659602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e6c04-1405-4f3f-87d5-84643ae3f228 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:06.884332 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-7d2e6c04-1405-4f3f-87d5-84643ae3f228 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b586d4b3b5a4739b6d81e7ed0e62fc6', 'request_id': 'req-7d2e6c04-1405-4f3f-87d5-84643ae3f228'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:07.021198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-7d2e6c04-1405-4f3f-87d5-84643ae3f228 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b586d4b3b5a4739b6d81e7ed0e62fc6', 'request_id': 'req-7d2e6c04-1405-4f3f-87d5-84643ae3f228'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:07.030253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-7d2e6c04-1405-4f3f-87d5-84643ae3f228 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b586d4b3b5a4739b6d81e7ed0e62fc6', 'request_id': 'req-7d2e6c04-1405-4f3f-87d5-84643ae3f228'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:07.060892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-7d2e6c04-1405-4f3f-87d5-84643ae3f228 admin admin] The token cache is being invalidate because project 2b586d4b3b5a4739b6d81e7ed0e62fc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:54:07.060892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-7d2e6c04-1405-4f3f-87d5-84643ae3f228 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:07.060892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-7d2e6c04-1405-4f3f-87d5-84643ae3f228 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:07.060892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1074/2134] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:54:06 2022] DELETE /identity/v3/projects/2b586d4b3b5a4739b6d81e7ed0e62fc6 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:54:07.061553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93550978-bdec-4d2f-b7fe-4d6be4292039 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:07.061553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93550978-bdec-4d2f-b7fe-4d6be4292039 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:07.061553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93550978-bdec-4d2f-b7fe-4d6be4292039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:07.096841 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-93550978-bdec-4d2f-b7fe-4d6be4292039 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:07.098881 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-93550978-bdec-4d2f-b7fe-4d6be4292039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:07.188298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1061/2135] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:07.198247 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52edbfe-0eae-427a-af29-53892c14249c None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:07.199437 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52edbfe-0eae-427a-af29-53892c14249c None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:07.200060 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52edbfe-0eae-427a-af29-53892c14249c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:07.230213 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-c52edbfe-0eae-427a-af29-53892c14249c None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:07.233140 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-c52edbfe-0eae-427a-af29-53892c14249c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:07.240940 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1075/2136] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:07.245597 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93550978-bdec-4d2f-b7fe-4d6be4292039 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:07.262376 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:07.266372 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93550978-bdec-4d2f-b7fe-4d6be4292039 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:07.271990 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7edccb-4215-41af-aa82-9d1255123205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:07.272634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7edccb-4215-41af-aa82-9d1255123205 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:07.272915 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7edccb-4215-41af-aa82-9d1255123205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:07.273205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7edccb-4215-41af-aa82-9d1255123205 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:07.273855 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7edccb-4215-41af-aa82-9d1255123205 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:54:07.274156 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7edccb-4215-41af-aa82-9d1255123205 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:07.275980 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7edccb-4215-41af-aa82-9d1255123205 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:07.304847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1062/2137] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:54:07.305547 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52edbfe-0eae-427a-af29-53892c14249c None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:07.320789 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52edbfe-0eae-427a-af29-53892c14249c None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:07.333403 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9281d0-9e79-4fe2-ac37-6e9a188f1e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:07.334614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9281d0-9e79-4fe2-ac37-6e9a188f1e0f admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:07.335216 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9281d0-9e79-4fe2-ac37-6e9a188f1e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:07.335730 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9281d0-9e79-4fe2-ac37-6e9a188f1e0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:07.336929 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9281d0-9e79-4fe2-ac37-6e9a188f1e0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:07.340814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9281d0-9e79-4fe2-ac37-6e9a188f1e0f admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:07.511388 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1076/2138] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:54:07 2022] POST /identity/v3/projects => generated 381 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:54:07.517337 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7edccb-4215-41af-aa82-9d1255123205 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:07.530157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7edccb-4215-41af-aa82-9d1255123205 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:07.566306 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68afad-6767-418e-8efb-26ee1b8996c3 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:07.566828 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68afad-6767-418e-8efb-26ee1b8996c3 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:07.567127 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68afad-6767-418e-8efb-26ee1b8996c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:07.595753 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-2f68afad-6767-418e-8efb-26ee1b8996c3 None None] MFA Rules not processed for user `d6d78cbc233a498db3a9268a4b93d737`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:07.598198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-2f68afad-6767-418e-8efb-26ee1b8996c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:07.646906 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ef20f-b2a4-462f-94ae-8b79d004474a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:07.647551 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ef20f-b2a4-462f-94ae-8b79d004474a admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:07.647831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ef20f-b2a4-462f-94ae-8b79d004474a admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:07.648097 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ef20f-b2a4-462f-94ae-8b79d004474a admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:07.648756 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ef20f-b2a4-462f-94ae-8b79d004474a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:07.650896 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ef20f-b2a4-462f-94ae-8b79d004474a admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:07.684341 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1077/2139] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:54:07.773712 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1063/2140] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:54:07 2022] POST /identity/v3/users => generated 291 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:54:07.783590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68afad-6767-418e-8efb-26ee1b8996c3 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:07.796606 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68afad-6767-418e-8efb-26ee1b8996c3 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:07.803701 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd88a12-e931-4e94-973a-eba1ba3be11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:07.804503 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd88a12-e931-4e94-973a-eba1ba3be11d admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:07.804864 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd88a12-e931-4e94-973a-eba1ba3be11d admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:07.805232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd88a12-e931-4e94-973a-eba1ba3be11d admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:07.806147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd88a12-e931-4e94-973a-eba1ba3be11d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:07.808132 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd88a12-e931-4e94-973a-eba1ba3be11d admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:07.823537 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1078/2141] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:54:07.829350 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ef20f-b2a4-462f-94ae-8b79d004474a admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:07.849624 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ef20f-b2a4-462f-94ae-8b79d004474a admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:07.868380 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8437692-2f76-4b5f-aafc-3aa0829a0f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:07.869078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8437692-2f76-4b5f-aafc-3aa0829a0f0e admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:07.869355 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8437692-2f76-4b5f-aafc-3aa0829a0f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:07.869625 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8437692-2f76-4b5f-aafc-3aa0829a0f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:07.870281 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8437692-2f76-4b5f-aafc-3aa0829a0f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:07.872024 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8437692-2f76-4b5f-aafc-3aa0829a0f0e admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:07.891419 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1064/2142] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:54:07.900064 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd88a12-e931-4e94-973a-eba1ba3be11d admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:07.917712 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd88a12-e931-4e94-973a-eba1ba3be11d admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:07.946716 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:07.955071 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734b4c1-6168-4aa9-911e-07abb2ab41e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:07.958014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734b4c1-6168-4aa9-911e-07abb2ab41e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:07.958014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734b4c1-6168-4aa9-911e-07abb2ab41e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:07.958014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734b4c1-6168-4aa9-911e-07abb2ab41e8 admin admin] PATH_INFO: `/v3/projects/49844b73a63b4c338fc5109f70ba774e/users/683bed0160e54791a8e638523ac7e460/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:07.991685 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734b4c1-6168-4aa9-911e-07abb2ab41e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49844b73a63b4c338fc5109f70ba774e, user_id=683bed0160e54791a8e638523ac7e460, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:07.998545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734b4c1-6168-4aa9-911e-07abb2ab41e8 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:08.090062 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1079/2143] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:54:07 2022] PUT /identity/v3/projects/49844b73a63b4c338fc5109f70ba774e/users/683bed0160e54791a8e638523ac7e460/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:54:08.098292 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a2a85-f136-4396-b5cf-2e6e8eb72bb7 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:08.099077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a2a85-f136-4396-b5cf-2e6e8eb72bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:08.099077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a2a85-f136-4396-b5cf-2e6e8eb72bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:08.129173 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-7f4a2a85-f136-4396-b5cf-2e6e8eb72bb7 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:08.130825 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-7f4a2a85-f136-4396-b5cf-2e6e8eb72bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:08.180467 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:08.257650 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1065/2144] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:54:08.266988 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734b4c1-6168-4aa9-911e-07abb2ab41e8 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:08.270923 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-2734b4c1-6168-4aa9-911e-07abb2ab41e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:08.341247 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734b4c1-6168-4aa9-911e-07abb2ab41e8 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:08.345311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2992b-8c56-4bad-9b2d-9fd07c9188c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:08.345927 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2992b-8c56-4bad-9b2d-9fd07c9188c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:08.346826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2992b-8c56-4bad-9b2d-9fd07c9188c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:08.347241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2992b-8c56-4bad-9b2d-9fd07c9188c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:08.358596 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e2992b-8c56-4bad-9b2d-9fd07c9188c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:08.360546 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e2992b-8c56-4bad-9b2d-9fd07c9188c1 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:08.373609 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1080/2145] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:54:09.879246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:10.516380 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599ca1c-dbb3-4e68-9e4e-b46ab680cdf0 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:10.516948 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599ca1c-dbb3-4e68-9e4e-b46ab680cdf0 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:10.517062 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599ca1c-dbb3-4e68-9e4e-b46ab680cdf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:10.573103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-1599ca1c-dbb3-4e68-9e4e-b46ab680cdf0 None None] MFA Rules not processed for user `683bed0160e54791a8e638523ac7e460`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:10.575747 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-1599ca1c-dbb3-4e68-9e4e-b46ab680cdf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:10.629587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:10.684048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1066/2146] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:54:10.701315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2992b-8c56-4bad-9b2d-9fd07c9188c1 service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:10.703365 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-48e2992b-8c56-4bad-9b2d-9fd07c9188c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:10.805305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2992b-8c56-4bad-9b2d-9fd07c9188c1 service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:10.809387 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9658f1b-6d15-49fb-b75d-5e3edb232ee3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:10.810142 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9658f1b-6d15-49fb-b75d-5e3edb232ee3 service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:10.810511 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9658f1b-6d15-49fb-b75d-5e3edb232ee3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:10.810913 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9658f1b-6d15-49fb-b75d-5e3edb232ee3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:10.821906 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9658f1b-6d15-49fb-b75d-5e3edb232ee3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:10.823386 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9658f1b-6d15-49fb-b75d-5e3edb232ee3 service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:10.835324 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1081/2147] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:54:12.996943 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d3ee6-b840-4264-980f-52c3c358d940 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:12.997750 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d3ee6-b840-4264-980f-52c3c358d940 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:12.997750 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d3ee6-b840-4264-980f-52c3c358d940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:13.022511 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-591d3ee6-b840-4264-980f-52c3c358d940 None None] MFA Rules not processed for user `683bed0160e54791a8e638523ac7e460`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:13.024668 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-591d3ee6-b840-4264-980f-52c3c358d940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:13.034701 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1067/2148] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:54:15.803683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9658f1b-6d15-49fb-b75d-5e3edb232ee3 service cinder] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:15.805208 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-a9658f1b-6d15-49fb-b75d-5e3edb232ee3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:15.848998 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9658f1b-6d15-49fb-b75d-5e3edb232ee3 service cinder] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:15.861789 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36804a-b559-4a4f-84bb-d9efcb463571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:15.865390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36804a-b559-4a4f-84bb-d9efcb463571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:15.865889 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36804a-b559-4a4f-84bb-d9efcb463571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:15.866552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36804a-b559-4a4f-84bb-d9efcb463571 admin admin] PATH_INFO: `/v3/users/9f32d71a19e745f19e70c60c89e04db7` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:15.872796 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36804a-b559-4a4f-84bb-d9efcb463571 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f32d71a19e745f19e70c60c89e04db7)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:15.878762 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36804a-b559-4a4f-84bb-d9efcb463571 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:16.182559 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-ed36804a-b559-4a4f-84bb-d9efcb463571 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f32d71a19e745f19e70c60c89e04db7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:16.284226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-ed36804a-b559-4a4f-84bb-d9efcb463571 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f32d71a19e745f19e70c60c89e04db7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:16.285682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-ed36804a-b559-4a4f-84bb-d9efcb463571 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f32d71a19e745f19e70c60c89e04db7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:16.300441 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-ed36804a-b559-4a4f-84bb-d9efcb463571 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f32d71a19e745f19e70c60c89e04db7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:16.341773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-ed36804a-b559-4a4f-84bb-d9efcb463571 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f32d71a19e745f19e70c60c89e04db7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:16.354550 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1082/2149] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:54:15 2022] DELETE /identity/v3/users/9f32d71a19e745f19e70c60c89e04db7 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:54:16.361131 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07851e-69b9-41d0-9bfe-2fa51efbe585 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:16.379132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07851e-69b9-41d0-9bfe-2fa51efbe585 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:16.379132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07851e-69b9-41d0-9bfe-2fa51efbe585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:16.430940 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-fe07851e-69b9-41d0-9bfe-2fa51efbe585 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:16.430940 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-fe07851e-69b9-41d0-9bfe-2fa51efbe585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:16.534700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1068/2150] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:16.544026 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36804a-b559-4a4f-84bb-d9efcb463571 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:16.546357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ed36804a-b559-4a4f-84bb-d9efcb463571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:16.615144 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:16.635644 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36804a-b559-4a4f-84bb-d9efcb463571 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:16.648488 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533b0f9-98f5-455b-94da-bdadd9010e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:16.649792 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533b0f9-98f5-455b-94da-bdadd9010e2b service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:16.650427 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533b0f9-98f5-455b-94da-bdadd9010e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:16.651196 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533b0f9-98f5-455b-94da-bdadd9010e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:16.669733 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533b0f9-98f5-455b-94da-bdadd9010e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:16.672208 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533b0f9-98f5-455b-94da-bdadd9010e2b service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:16.710218 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1083/2151] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:54:17.672799 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07851e-69b9-41d0-9bfe-2fa51efbe585 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:17.674481 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-fe07851e-69b9-41d0-9bfe-2fa51efbe585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:17.695070 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07851e-69b9-41d0-9bfe-2fa51efbe585 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:17.701875 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17abc020-30b6-4262-bdb3-be6e3974130f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:17.702735 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abc020-30b6-4262-bdb3-be6e3974130f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:17.703150 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abc020-30b6-4262-bdb3-be6e3974130f admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:17.703583 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abc020-30b6-4262-bdb3-be6e3974130f admin admin] PATH_INFO: `/v3/projects/7cafcc6cb5c749e297eea217a0e53f94` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:17.706049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17abc020-30b6-4262-bdb3-be6e3974130f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cafcc6cb5c749e297eea217a0e53f94)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:17.708352 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17abc020-30b6-4262-bdb3-be6e3974130f admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:17.861014 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-17abc020-30b6-4262-bdb3-be6e3974130f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cafcc6cb5c749e297eea217a0e53f94', 'request_id': 'req-17abc020-30b6-4262-bdb3-be6e3974130f'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:17.882187 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-17abc020-30b6-4262-bdb3-be6e3974130f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cafcc6cb5c749e297eea217a0e53f94', 'request_id': 'req-17abc020-30b6-4262-bdb3-be6e3974130f'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:17.891301 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-17abc020-30b6-4262-bdb3-be6e3974130f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cafcc6cb5c749e297eea217a0e53f94', 'request_id': 'req-17abc020-30b6-4262-bdb3-be6e3974130f'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:17.976970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-17abc020-30b6-4262-bdb3-be6e3974130f admin admin] The token cache is being invalidate because project 7cafcc6cb5c749e297eea217a0e53f94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:54:17.977627 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-17abc020-30b6-4262-bdb3-be6e3974130f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:17.978480 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-17abc020-30b6-4262-bdb3-be6e3974130f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:17.980654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1069/2152] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:54:17 2022] DELETE /identity/v3/projects/7cafcc6cb5c749e297eea217a0e53f94 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:54:17.985024 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533b0f9-98f5-455b-94da-bdadd9010e2b service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:17.996188 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533b0f9-98f5-455b-94da-bdadd9010e2b service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:18.074209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47a83a-3aeb-4447-b27a-ac8e25e6d8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:18.074917 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47a83a-3aeb-4447-b27a-ac8e25e6d8ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:18.075200 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47a83a-3aeb-4447-b27a-ac8e25e6d8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:18.075449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47a83a-3aeb-4447-b27a-ac8e25e6d8ba admin admin] PATH_INFO: `/v3/users/c481bedc075342aaae758e0487726ef1` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:18.077374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b47a83a-3aeb-4447-b27a-ac8e25e6d8ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=c481bedc075342aaae758e0487726ef1)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:18.079931 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b47a83a-3aeb-4447-b27a-ac8e25e6d8ba admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:18.349861 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3b47a83a-3aeb-4447-b27a-ac8e25e6d8ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c481bedc075342aaae758e0487726ef1'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:18.462094 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3b47a83a-3aeb-4447-b27a-ac8e25e6d8ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c481bedc075342aaae758e0487726ef1'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:18.464916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3b47a83a-3aeb-4447-b27a-ac8e25e6d8ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c481bedc075342aaae758e0487726ef1'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:18.481231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3b47a83a-3aeb-4447-b27a-ac8e25e6d8ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c481bedc075342aaae758e0487726ef1'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:18.508936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3b47a83a-3aeb-4447-b27a-ac8e25e6d8ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c481bedc075342aaae758e0487726ef1'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:18.523120 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1084/2153] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:54:17 2022] DELETE /identity/v3/users/c481bedc075342aaae758e0487726ef1 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:54:19.716105 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17abc020-30b6-4262-bdb3-be6e3974130f admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:19.718749 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-17abc020-30b6-4262-bdb3-be6e3974130f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:19.746884 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17abc020-30b6-4262-bdb3-be6e3974130f admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:19.856276 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:19.941603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00bb8f-eeb3-4f37-be42-92d7165186dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:19.942551 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00bb8f-eeb3-4f37-be42-92d7165186dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:19.943034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00bb8f-eeb3-4f37-be42-92d7165186dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:19.944107 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00bb8f-eeb3-4f37-be42-92d7165186dc admin admin] PATH_INFO: `/v3/projects/b69b9d9c02794402bb1c0aacf686c87e` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:19.957212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be00bb8f-eeb3-4f37-be42-92d7165186dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b69b9d9c02794402bb1c0aacf686c87e)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:19.964297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be00bb8f-eeb3-4f37-be42-92d7165186dc admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:20.209858 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-be00bb8f-eeb3-4f37-be42-92d7165186dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b69b9d9c02794402bb1c0aacf686c87e', 'request_id': 'req-be00bb8f-eeb3-4f37-be42-92d7165186dc'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:20.240146 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-be00bb8f-eeb3-4f37-be42-92d7165186dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b69b9d9c02794402bb1c0aacf686c87e', 'request_id': 'req-be00bb8f-eeb3-4f37-be42-92d7165186dc'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:20.258468 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-be00bb8f-eeb3-4f37-be42-92d7165186dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b69b9d9c02794402bb1c0aacf686c87e', 'request_id': 'req-be00bb8f-eeb3-4f37-be42-92d7165186dc'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:20.386369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-be00bb8f-eeb3-4f37-be42-92d7165186dc admin admin] The token cache is being invalidate because project b69b9d9c02794402bb1c0aacf686c87e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:54:20.386702 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-be00bb8f-eeb3-4f37-be42-92d7165186dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:20.392280 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-be00bb8f-eeb3-4f37-be42-92d7165186dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:20.396864 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1070/2154] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:54:19 2022] DELETE /identity/v3/projects/b69b9d9c02794402bb1c0aacf686c87e => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:54:20.403744 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13214f4-56dd-45b5-99ce-a4b4a2f0273a None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:20.405282 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13214f4-56dd-45b5-99ce-a4b4a2f0273a None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:20.406777 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13214f4-56dd-45b5-99ce-a4b4a2f0273a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:20.484209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-a13214f4-56dd-45b5-99ce-a4b4a2f0273a None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:20.499066 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-a13214f4-56dd-45b5-99ce-a4b4a2f0273a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:20.630526 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:20.646124 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1085/2155] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:20.655774 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1393eb-28f3-4e06-9e91-f17c77732ed1 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:20.656113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1393eb-28f3-4e06-9e91-f17c77732ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:20.656366 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1393eb-28f3-4e06-9e91-f17c77732ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:20.697956 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-fe1393eb-28f3-4e06-9e91-f17c77732ed1 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:20.700102 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-fe1393eb-28f3-4e06-9e91-f17c77732ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:20.714080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1071/2156] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:54:20.719657 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13214f4-56dd-45b5-99ce-a4b4a2f0273a None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:20.735789 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13214f4-56dd-45b5-99ce-a4b4a2f0273a None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:20.749609 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693d58a-f91f-4c72-998c-06ada534609b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:20.751035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693d58a-f91f-4c72-998c-06ada534609b admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:20.752879 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693d58a-f91f-4c72-998c-06ada534609b admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:20.758526 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693d58a-f91f-4c72-998c-06ada534609b admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:20.758526 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5693d58a-f91f-4c72-998c-06ada534609b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:54:20.758935 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5693d58a-f91f-4c72-998c-06ada534609b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:20.762744 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5693d58a-f91f-4c72-998c-06ada534609b admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:20.786614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1086/2157] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:54:20.791078 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1393eb-28f3-4e06-9e91-f17c77732ed1 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:20.802090 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1393eb-28f3-4e06-9e91-f17c77732ed1 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:20.810064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557f062-8576-4a58-81dd-568e00a00b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:20.811029 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557f062-8576-4a58-81dd-568e00a00b54 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:20.811503 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557f062-8576-4a58-81dd-568e00a00b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:20.811936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557f062-8576-4a58-81dd-568e00a00b54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:20.812837 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0557f062-8576-4a58-81dd-568e00a00b54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:20.824373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0557f062-8576-4a58-81dd-568e00a00b54 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:21.069293 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1072/2158] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:54:20 2022] POST /identity/v3/projects => generated 367 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:54:21.075546 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693d58a-f91f-4c72-998c-06ada534609b admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:21.094345 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693d58a-f91f-4c72-998c-06ada534609b admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:21.215520 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6c891-87b2-46f9-8ad9-27ee3088d90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:21.216169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6c891-87b2-46f9-8ad9-27ee3088d90e admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:21.216461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6c891-87b2-46f9-8ad9-27ee3088d90e admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:21.216724 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6c891-87b2-46f9-8ad9-27ee3088d90e admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:21.217440 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d6c891-87b2-46f9-8ad9-27ee3088d90e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:21.219841 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d6c891-87b2-46f9-8ad9-27ee3088d90e admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:21.370099 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1087/2159] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:54:21 2022] POST /identity/v3/users => generated 284 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:54:21.372181 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557f062-8576-4a58-81dd-568e00a00b54 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:21.387140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557f062-8576-4a58-81dd-568e00a00b54 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:21.395638 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6046255-f64b-4847-84e7-abb475bd1603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:21.396722 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6046255-f64b-4847-84e7-abb475bd1603 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:21.397234 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6046255-f64b-4847-84e7-abb475bd1603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:21.397691 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6046255-f64b-4847-84e7-abb475bd1603 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:21.398922 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6046255-f64b-4847-84e7-abb475bd1603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:21.402151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6046255-f64b-4847-84e7-abb475bd1603 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:21.421953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1073/2160] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:54:21.427037 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6c891-87b2-46f9-8ad9-27ee3088d90e admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:21.440734 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6c891-87b2-46f9-8ad9-27ee3088d90e admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:21.446878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a369d-501c-4be3-b954-9332589fbd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:21.447737 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a369d-501c-4be3-b954-9332589fbd05 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:21.448590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a369d-501c-4be3-b954-9332589fbd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:21.448985 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a369d-501c-4be3-b954-9332589fbd05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:21.449787 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a369d-501c-4be3-b954-9332589fbd05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:21.451897 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a369d-501c-4be3-b954-9332589fbd05 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:21.473625 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1088/2161] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:54:21.479153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6046255-f64b-4847-84e7-abb475bd1603 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:21.499217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6046255-f64b-4847-84e7-abb475bd1603 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:21.508977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104973b-55c7-4967-9daa-b11fecb4e532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:21.510018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104973b-55c7-4967-9daa-b11fecb4e532 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:21.510511 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104973b-55c7-4967-9daa-b11fecb4e532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:21.511155 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104973b-55c7-4967-9daa-b11fecb4e532 admin admin] PATH_INFO: `/v3/projects/0b47624c97294f708ae129c4c277cc78/users/9c0ca46f5d0546eb885444934f7a8a21/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:21.539324 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7104973b-55c7-4967-9daa-b11fecb4e532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b47624c97294f708ae129c4c277cc78, user_id=9c0ca46f5d0546eb885444934f7a8a21, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:21.545386 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7104973b-55c7-4967-9daa-b11fecb4e532 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:21.696987 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1074/2162] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:54:21 2022] PUT /identity/v3/projects/0b47624c97294f708ae129c4c277cc78/users/9c0ca46f5d0546eb885444934f7a8a21/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:54:21.708586 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce972740-8fbc-4685-93d7-ee1812db7d74 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:21.708926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce972740-8fbc-4685-93d7-ee1812db7d74 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:21.709225 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce972740-8fbc-4685-93d7-ee1812db7d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:21.744352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-ce972740-8fbc-4685-93d7-ee1812db7d74 None None] MFA Rules not processed for user `9c0ca46f5d0546eb885444934f7a8a21`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:21.745877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ce972740-8fbc-4685-93d7-ee1812db7d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:21.840624 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1089/2163] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:54:21.847201 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104973b-55c7-4967-9daa-b11fecb4e532 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:21.920437 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104973b-55c7-4967-9daa-b11fecb4e532 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:21.926979 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd34f24-e997-46de-a834-64fe0a89927c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:21.927615 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd34f24-e997-46de-a834-64fe0a89927c admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:21.927954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd34f24-e997-46de-a834-64fe0a89927c admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:21.928228 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd34f24-e997-46de-a834-64fe0a89927c admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:21.928881 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd34f24-e997-46de-a834-64fe0a89927c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:21.930900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd34f24-e997-46de-a834-64fe0a89927c admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:21.974721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce972740-8fbc-4685-93d7-ee1812db7d74 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:21.976228 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ce972740-8fbc-4685-93d7-ee1812db7d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:21.991350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce972740-8fbc-4685-93d7-ee1812db7d74 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:21.996122 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df02c8e1-45a6-44fe-a58b-d5ce11d12014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:21.997504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02c8e1-45a6-44fe-a58b-d5ce11d12014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:21.997504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02c8e1-45a6-44fe-a58b-d5ce11d12014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:21.997504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02c8e1-45a6-44fe-a58b-d5ce11d12014 admin admin] PATH_INFO: `/v3/users/eef585b05da2444cbe8b715a55a5bcc6` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:21.998722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df02c8e1-45a6-44fe-a58b-d5ce11d12014 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eef585b05da2444cbe8b715a55a5bcc6)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:22.000697 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df02c8e1-45a6-44fe-a58b-d5ce11d12014 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:22.096595 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:22.104940 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1075/2164] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:54:21 2022] POST /identity/v3/projects => generated 365 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:54:22.109957 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd34f24-e997-46de-a834-64fe0a89927c admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:22.208485 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd34f24-e997-46de-a834-64fe0a89927c admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:22.213465 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70d7d8-eaa4-4d84-a050-7ad5b3b908a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:22.214089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70d7d8-eaa4-4d84-a050-7ad5b3b908a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:22.214363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70d7d8-eaa4-4d84-a050-7ad5b3b908a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:22.214692 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70d7d8-eaa4-4d84-a050-7ad5b3b908a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:22.215350 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70d7d8-eaa4-4d84-a050-7ad5b3b908a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:22.217326 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70d7d8-eaa4-4d84-a050-7ad5b3b908a7 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:22.240663 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-df02c8e1-45a6-44fe-a58b-d5ce11d12014 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eef585b05da2444cbe8b715a55a5bcc6'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:22.313246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1076/2165] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:54:22 2022] POST /identity/v3/users => generated 289 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:54:22.318994 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70d7d8-eaa4-4d84-a050-7ad5b3b908a7 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:22.331168 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-df02c8e1-45a6-44fe-a58b-d5ce11d12014 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eef585b05da2444cbe8b715a55a5bcc6'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:22.332219 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-df02c8e1-45a6-44fe-a58b-d5ce11d12014 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eef585b05da2444cbe8b715a55a5bcc6'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:22.341622 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70d7d8-eaa4-4d84-a050-7ad5b3b908a7 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:22.345290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-df02c8e1-45a6-44fe-a58b-d5ce11d12014 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eef585b05da2444cbe8b715a55a5bcc6'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:22.352983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f719601-815b-4c6a-97e8-9bab5ead6f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:22.354248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f719601-815b-4c6a-97e8-9bab5ead6f90 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:22.354752 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f719601-815b-4c6a-97e8-9bab5ead6f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:22.355188 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f719601-815b-4c6a-97e8-9bab5ead6f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:22.356744 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f719601-815b-4c6a-97e8-9bab5ead6f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:22.359882 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f719601-815b-4c6a-97e8-9bab5ead6f90 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:22.387570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1077/2166] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:54:22.388912 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f719601-815b-4c6a-97e8-9bab5ead6f90 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:22.389164 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-df02c8e1-45a6-44fe-a58b-d5ce11d12014 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eef585b05da2444cbe8b715a55a5bcc6'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:22.391806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-2f719601-815b-4c6a-97e8-9bab5ead6f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:22.399153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1090/2167] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:54:21 2022] DELETE /identity/v3/users/eef585b05da2444cbe8b715a55a5bcc6 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:54:22.406691 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed643a-ddb8-4233-8b86-f62342203aba None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:22.407567 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed643a-ddb8-4233-8b86-f62342203aba None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:22.408844 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed643a-ddb8-4233-8b86-f62342203aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:22.420367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:22.421159 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f719601-815b-4c6a-97e8-9bab5ead6f90 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:22.448793 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-e4ed643a-ddb8-4233-8b86-f62342203aba None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:22.450404 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-e4ed643a-ddb8-4233-8b86-f62342203aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:22.526309 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a6dcb-0b0f-4e6f-98ec-fb88aac0b19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:22.527588 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a6dcb-0b0f-4e6f-98ec-fb88aac0b19d admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:22.528094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a6dcb-0b0f-4e6f-98ec-fb88aac0b19d admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:22.528640 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a6dcb-0b0f-4e6f-98ec-fb88aac0b19d admin admin] PATH_INFO: `/v3/projects/2dec2bce436c42d397b5c41e61538925/users/7c976366f7ea4480bf2d69025e0862a7/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:22.532666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1091/2168] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:54:22.547643 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed643a-ddb8-4233-8b86-f62342203aba None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:22.549498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-e4ed643a-ddb8-4233-8b86-f62342203aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:22.555629 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a6dcb-0b0f-4e6f-98ec-fb88aac0b19d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dec2bce436c42d397b5c41e61538925, user_id=7c976366f7ea4480bf2d69025e0862a7, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:22.558735 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a6dcb-0b0f-4e6f-98ec-fb88aac0b19d admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:22.645117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed643a-ddb8-4233-8b86-f62342203aba None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:22.657168 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc0605b-ff33-49f2-b324-e927e3b97d63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:22.658717 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0605b-ff33-49f2-b324-e927e3b97d63 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:22.659213 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0605b-ff33-49f2-b324-e927e3b97d63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:22.659555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0605b-ff33-49f2-b324-e927e3b97d63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:22.667449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1078/2169] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:54:22 2022] PUT /identity/v3/projects/2dec2bce436c42d397b5c41e61538925/users/7c976366f7ea4480bf2d69025e0862a7/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:54:22.669583 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a6dcb-0b0f-4e6f-98ec-fb88aac0b19d admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:22.671758 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc0605b-ff33-49f2-b324-e927e3b97d63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:22.677234 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc0605b-ff33-49f2-b324-e927e3b97d63 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:22.689054 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1092/2170] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:54:22.777151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a6dcb-0b0f-4e6f-98ec-fb88aac0b19d admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:22.786586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54db507-e1a2-47ca-9e84-76a0c298f166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:22.787559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54db507-e1a2-47ca-9e84-76a0c298f166 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:22.788283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54db507-e1a2-47ca-9e84-76a0c298f166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:22.788493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54db507-e1a2-47ca-9e84-76a0c298f166 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:22.789438 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54db507-e1a2-47ca-9e84-76a0c298f166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:22.791796 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54db507-e1a2-47ca-9e84-76a0c298f166 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:22.807369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1079/2171] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:54:22.811964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc0605b-ff33-49f2-b324-e927e3b97d63 service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:22.828111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc0605b-ff33-49f2-b324-e927e3b97d63 service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:22.837098 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae652112-eea9-4b55-8619-9e123e38c7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:22.837788 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae652112-eea9-4b55-8619-9e123e38c7fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:22.838086 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae652112-eea9-4b55-8619-9e123e38c7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:22.838332 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae652112-eea9-4b55-8619-9e123e38c7fd admin admin] PATH_INFO: `/v3/projects/2dec2bce436c42d397b5c41e61538925/users/7c976366f7ea4480bf2d69025e0862a7/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:22.840558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae652112-eea9-4b55-8619-9e123e38c7fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dec2bce436c42d397b5c41e61538925, user_id=7c976366f7ea4480bf2d69025e0862a7, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:22.849138 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae652112-eea9-4b55-8619-9e123e38c7fd admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:22.866666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1093/2172] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:54:22 2022] PUT /identity/v3/projects/2dec2bce436c42d397b5c41e61538925/users/7c976366f7ea4480bf2d69025e0862a7/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:54:22.877142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab255b6-60c4-40a8-8d50-93a70f67ad64 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:22.878092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab255b6-60c4-40a8-8d50-93a70f67ad64 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:22.878092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab255b6-60c4-40a8-8d50-93a70f67ad64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:22.937319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-4ab255b6-60c4-40a8-8d50-93a70f67ad64 None None] MFA Rules not processed for user `7c976366f7ea4480bf2d69025e0862a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:22.938695 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-4ab255b6-60c4-40a8-8d50-93a70f67ad64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:23.059740 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1080/2173] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:22 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:23.078620 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae652112-eea9-4b55-8619-9e123e38c7fd admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:23.080845 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ae652112-eea9-4b55-8619-9e123e38c7fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:23.106973 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:23.160574 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae652112-eea9-4b55-8619-9e123e38c7fd admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:23.167668 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2948517-6553-4dc6-9844-fc61be1ffe1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:23.168815 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2948517-6553-4dc6-9844-fc61be1ffe1c service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:23.168815 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2948517-6553-4dc6-9844-fc61be1ffe1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:23.169221 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2948517-6553-4dc6-9844-fc61be1ffe1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:23.193875 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2948517-6553-4dc6-9844-fc61be1ffe1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:23.195716 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2948517-6553-4dc6-9844-fc61be1ffe1c service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:23.207907 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:23.214081 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1094/2174] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:23 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:54:23.647165 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab255b6-60c4-40a8-8d50-93a70f67ad64 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:23.667346 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab255b6-60c4-40a8-8d50-93a70f67ad64 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:23.674178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf67490-1c4e-42ca-b8a9-ddad81df210f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:23.675714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf67490-1c4e-42ca-b8a9-ddad81df210f service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:23.676188 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf67490-1c4e-42ca-b8a9-ddad81df210f service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:23.676630 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf67490-1c4e-42ca-b8a9-ddad81df210f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:23.680804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ddf67490-1c4e-42ca-b8a9-ddad81df210f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:23.722879 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2948517-6553-4dc6-9844-fc61be1ffe1c service cinder] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:23.731232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-e2948517-6553-4dc6-9844-fc61be1ffe1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:23.871367 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf67490-1c4e-42ca-b8a9-ddad81df210f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:23.873525 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf67490-1c4e-42ca-b8a9-ddad81df210f service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:23.887775 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1081/2175] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:54:23.927512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2948517-6553-4dc6-9844-fc61be1ffe1c service cinder] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:23.936557 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d770ad-9153-4dab-9410-ae76d8c59afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:23.937567 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d770ad-9153-4dab-9410-ae76d8c59afe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:23.938006 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d770ad-9153-4dab-9410-ae76d8c59afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:23.938425 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d770ad-9153-4dab-9410-ae76d8c59afe admin admin] PATH_INFO: `/v3/projects/e475c58bc22940dea4bc85a183ac7053` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:23.940681 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d770ad-9153-4dab-9410-ae76d8c59afe admin admin] RBAC: Authorizing `identity:delete_project(project_id=e475c58bc22940dea4bc85a183ac7053)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:23.943722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d770ad-9153-4dab-9410-ae76d8c59afe admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:24.117258 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-c6d770ad-9153-4dab-9410-ae76d8c59afe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e475c58bc22940dea4bc85a183ac7053', 'request_id': 'req-c6d770ad-9153-4dab-9410-ae76d8c59afe'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:24.252078 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-c6d770ad-9153-4dab-9410-ae76d8c59afe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e475c58bc22940dea4bc85a183ac7053', 'request_id': 'req-c6d770ad-9153-4dab-9410-ae76d8c59afe'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:24.285824 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-c6d770ad-9153-4dab-9410-ae76d8c59afe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e475c58bc22940dea4bc85a183ac7053', 'request_id': 'req-c6d770ad-9153-4dab-9410-ae76d8c59afe'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:24.334233 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-c6d770ad-9153-4dab-9410-ae76d8c59afe admin admin] The token cache is being invalidate because project e475c58bc22940dea4bc85a183ac7053 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:54:24.335497 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-c6d770ad-9153-4dab-9410-ae76d8c59afe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:24.350307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-c6d770ad-9153-4dab-9410-ae76d8c59afe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:24.354554 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1095/2176] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:54:23 2022] DELETE /identity/v3/projects/e475c58bc22940dea4bc85a183ac7053 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:54:24.367882 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc0db2-1031-4f5a-b77e-d5b6c4e08498 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:24.368563 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc0db2-1031-4f5a-b77e-d5b6c4e08498 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:24.371146 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc0db2-1031-4f5a-b77e-d5b6c4e08498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:24.422134 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-54bc0db2-1031-4f5a-b77e-d5b6c4e08498 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:24.422134 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-54bc0db2-1031-4f5a-b77e-d5b6c4e08498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:24.511772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1082/2177] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:24.523511 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c42483-81d3-4965-b5b5-5ff93920450f None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:24.523967 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c42483-81d3-4965-b5b5-5ff93920450f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:24.524476 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c42483-81d3-4965-b5b5-5ff93920450f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:24.550987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-03c42483-81d3-4965-b5b5-5ff93920450f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:24.553366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-03c42483-81d3-4965-b5b5-5ff93920450f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:24.576501 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1096/2178] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:54:24.578256 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc0db2-1031-4f5a-b77e-d5b6c4e08498 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:24.593137 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc0db2-1031-4f5a-b77e-d5b6c4e08498 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:24.623203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2b0f4-5d03-47c4-b6a7-1965b48db12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:24.624254 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2b0f4-5d03-47c4-b6a7-1965b48db12b admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:24.624855 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2b0f4-5d03-47c4-b6a7-1965b48db12b admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:24.625582 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2b0f4-5d03-47c4-b6a7-1965b48db12b admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:24.627503 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd2b0f4-5d03-47c4-b6a7-1965b48db12b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:54:24.629864 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd2b0f4-5d03-47c4-b6a7-1965b48db12b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:24.637020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd2b0f4-5d03-47c4-b6a7-1965b48db12b admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:24.680571 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1083/2179] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:54:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:54:24.682429 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c42483-81d3-4965-b5b5-5ff93920450f None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:24.703330 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c42483-81d3-4965-b5b5-5ff93920450f None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:24.748148 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4c725-504c-4982-82bb-001d9bdc7cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:24.749201 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4c725-504c-4982-82bb-001d9bdc7cea admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:24.749201 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4c725-504c-4982-82bb-001d9bdc7cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:24.749525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4c725-504c-4982-82bb-001d9bdc7cea admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:24.750142 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad4c725-504c-4982-82bb-001d9bdc7cea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:24.753224 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad4c725-504c-4982-82bb-001d9bdc7cea admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:24.908804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2b0f4-5d03-47c4-b6a7-1965b48db12b admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:24.909677 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1097/2180] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:54:24 2022] POST /identity/v3/projects => generated 399 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:54:24.945114 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2b0f4-5d03-47c4-b6a7-1965b48db12b admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:24.974060 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:25.138925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023e534-f0c4-44cf-8d93-57afc13beec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:25.139680 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023e534-f0c4-44cf-8d93-57afc13beec0 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:25.140088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023e534-f0c4-44cf-8d93-57afc13beec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:25.140475 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023e534-f0c4-44cf-8d93-57afc13beec0 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:25.141726 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023e534-f0c4-44cf-8d93-57afc13beec0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:25.145502 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023e534-f0c4-44cf-8d93-57afc13beec0 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:25.333012 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4c725-504c-4982-82bb-001d9bdc7cea admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:25.335088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1084/2181] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:54:24 2022] POST /identity/v3/users => generated 300 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:54:25.348188 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4c725-504c-4982-82bb-001d9bdc7cea admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:25.354485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c101fb-5673-407b-99db-2f0e343f228d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:25.355892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c101fb-5673-407b-99db-2f0e343f228d admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:25.357766 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c101fb-5673-407b-99db-2f0e343f228d admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:25.358338 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c101fb-5673-407b-99db-2f0e343f228d admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:25.359543 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c101fb-5673-407b-99db-2f0e343f228d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:25.362345 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c101fb-5673-407b-99db-2f0e343f228d admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:25.383145 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023e534-f0c4-44cf-8d93-57afc13beec0 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:25.399686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1098/2182] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:54:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:54:25.432930 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023e534-f0c4-44cf-8d93-57afc13beec0 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:25.450773 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e51b6-0367-4185-a8d1-7bbc007887e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:25.453985 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e51b6-0367-4185-a8d1-7bbc007887e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:25.454513 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e51b6-0367-4185-a8d1-7bbc007887e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:25.457963 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e51b6-0367-4185-a8d1-7bbc007887e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:25.458999 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e51b6-0367-4185-a8d1-7bbc007887e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:25.460869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e51b6-0367-4185-a8d1-7bbc007887e5 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:25.500041 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1085/2183] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:54:25 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:54:25.500995 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c101fb-5673-407b-99db-2f0e343f228d admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:25.547867 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c101fb-5673-407b-99db-2f0e343f228d admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:25.556574 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88c230-fa69-4bf2-944d-a5b6b3c1daf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:25.557713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88c230-fa69-4bf2-944d-a5b6b3c1daf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:25.558226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88c230-fa69-4bf2-944d-a5b6b3c1daf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:25.558879 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88c230-fa69-4bf2-944d-a5b6b3c1daf4 admin admin] PATH_INFO: `/v3/projects/18512864f4d942b69c471e2b7e6d77ff/users/9d7496c6be154ac4906855191667c7ff/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:25.581700 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88c230-fa69-4bf2-944d-a5b6b3c1daf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18512864f4d942b69c471e2b7e6d77ff, user_id=9d7496c6be154ac4906855191667c7ff, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:25.585138 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88c230-fa69-4bf2-944d-a5b6b3c1daf4 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:25.732892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1099/2184] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:54:25 2022] PUT /identity/v3/projects/18512864f4d942b69c471e2b7e6d77ff/users/9d7496c6be154ac4906855191667c7ff/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:54:25.746829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c97814-32c2-487f-bf2e-572ea7a95216 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:25.747669 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c97814-32c2-487f-bf2e-572ea7a95216 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:25.748106 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c97814-32c2-487f-bf2e-572ea7a95216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:25.791975 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-d4c97814-32c2-487f-bf2e-572ea7a95216 None None] MFA Rules not processed for user `9d7496c6be154ac4906855191667c7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:25.794062 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d4c97814-32c2-487f-bf2e-572ea7a95216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:25.917243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1086/2185] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:25 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:54:25.935967 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88c230-fa69-4bf2-944d-a5b6b3c1daf4 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:25.942102 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-9c88c230-fa69-4bf2-944d-a5b6b3c1daf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:26.006609 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:26.054957 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88c230-fa69-4bf2-944d-a5b6b3c1daf4 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:26.061287 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84366eb-802a-476a-a582-dc96be44864c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:26.062343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84366eb-802a-476a-a582-dc96be44864c service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:26.063172 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84366eb-802a-476a-a582-dc96be44864c service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:26.063688 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84366eb-802a-476a-a582-dc96be44864c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:26.076113 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84366eb-802a-476a-a582-dc96be44864c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:26.082778 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84366eb-802a-476a-a582-dc96be44864c service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:26.089296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1100/2186] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:25 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:54:32.849834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f4c56-c025-406d-87ef-71fb094badf0 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:32.858009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f4c56-c025-406d-87ef-71fb094badf0 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:32.858802 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f4c56-c025-406d-87ef-71fb094badf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:32.918128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-fa5f4c56-c025-406d-87ef-71fb094badf0 None None] MFA Rules not processed for user `9646662b0b3048f79ad5d2b9602f10a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:32.921679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-fa5f4c56-c025-406d-87ef-71fb094badf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:32.966224 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84366eb-802a-476a-a582-dc96be44864c service cinder] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:32.968457 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b84366eb-802a-476a-a582-dc96be44864c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:33.025475 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1087/2187] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Aug 8 16:54:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:33.109754 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84366eb-802a-476a-a582-dc96be44864c service cinder] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:33.113977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f4c56-c025-406d-87ef-71fb094badf0 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:33.116817 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-fa5f4c56-c025-406d-87ef-71fb094badf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:33.117174 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e6451-7e25-4745-8bee-510a18b1d2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:33.118045 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e6451-7e25-4745-8bee-510a18b1d2b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:33.118416 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e6451-7e25-4745-8bee-510a18b1d2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:33.118997 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e6451-7e25-4745-8bee-510a18b1d2b4 admin admin] PATH_INFO: `/v3/users/9c0ca46f5d0546eb885444934f7a8a21` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:33.120427 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e6451-7e25-4745-8bee-510a18b1d2b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c0ca46f5d0546eb885444934f7a8a21)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:33.123044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e6451-7e25-4745-8bee-510a18b1d2b4 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:33.160999 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:33.221544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f4c56-c025-406d-87ef-71fb094badf0 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:33.231545 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c181206-7f5e-412e-a590-b45694345f60 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c606decb832444e38a194b98f8d15f8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:33.232655 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c181206-7f5e-412e-a590-b45694345f60 service swift] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:33.233094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c181206-7f5e-412e-a590-b45694345f60 service swift] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:33.233559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c181206-7f5e-412e-a590-b45694345f60 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:33.251728 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c181206-7f5e-412e-a590-b45694345f60 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:33.255556 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c181206-7f5e-412e-a590-b45694345f60 service swift] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:33.293659 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1088/2188] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Aug 8 16:54:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 16:54:33.467212 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5e7e6451-7e25-4745-8bee-510a18b1d2b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c0ca46f5d0546eb885444934f7a8a21'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:33.573547 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5e7e6451-7e25-4745-8bee-510a18b1d2b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c0ca46f5d0546eb885444934f7a8a21'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:33.575300 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5e7e6451-7e25-4745-8bee-510a18b1d2b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c0ca46f5d0546eb885444934f7a8a21'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:33.615236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5e7e6451-7e25-4745-8bee-510a18b1d2b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c0ca46f5d0546eb885444934f7a8a21'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:33.651663 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5e7e6451-7e25-4745-8bee-510a18b1d2b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c0ca46f5d0546eb885444934f7a8a21'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:33.664520 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1101/2189] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:54:32 2022] DELETE /identity/v3/users/9c0ca46f5d0546eb885444934f7a8a21 => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:54:33.669993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049a351-792e-45ae-a800-7418b02a72e2 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:33.670953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049a351-792e-45ae-a800-7418b02a72e2 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:33.671835 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049a351-792e-45ae-a800-7418b02a72e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:33.728969 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-d049a351-792e-45ae-a800-7418b02a72e2 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:33.731702 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d049a351-792e-45ae-a800-7418b02a72e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:33.890839 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1089/2190] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:54:33.914790 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e6451-7e25-4745-8bee-510a18b1d2b4 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:33.917312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-5e7e6451-7e25-4745-8bee-510a18b1d2b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:34.072067 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:34.081458 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e6451-7e25-4745-8bee-510a18b1d2b4 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:34.086869 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d792264-aebb-4937-800d-81064b687de4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:34.087536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d792264-aebb-4937-800d-81064b687de4 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:34.087813 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d792264-aebb-4937-800d-81064b687de4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:34.088078 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d792264-aebb-4937-800d-81064b687de4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:34.101023 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d792264-aebb-4937-800d-81064b687de4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:34.103030 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d792264-aebb-4937-800d-81064b687de4 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:34.137190 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1102/2191] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:54:35.230065 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049a351-792e-45ae-a800-7418b02a72e2 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:35.236526 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d049a351-792e-45ae-a800-7418b02a72e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:35.256870 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049a351-792e-45ae-a800-7418b02a72e2 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:35.283252 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62808c-fcd8-40e9-95cd-d63c0cb615d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:35.284319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62808c-fcd8-40e9-95cd-d63c0cb615d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:35.284840 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62808c-fcd8-40e9-95cd-d63c0cb615d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:35.286088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62808c-fcd8-40e9-95cd-d63c0cb615d5 admin admin] PATH_INFO: `/v3/projects/0b47624c97294f708ae129c4c277cc78` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:35.287829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62808c-fcd8-40e9-95cd-d63c0cb615d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b47624c97294f708ae129c4c277cc78)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:35.290356 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62808c-fcd8-40e9-95cd-d63c0cb615d5 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:35.552974 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-db62808c-fcd8-40e9-95cd-d63c0cb615d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b47624c97294f708ae129c4c277cc78', 'request_id': 'req-db62808c-fcd8-40e9-95cd-d63c0cb615d5'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:35.569279 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-db62808c-fcd8-40e9-95cd-d63c0cb615d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b47624c97294f708ae129c4c277cc78', 'request_id': 'req-db62808c-fcd8-40e9-95cd-d63c0cb615d5'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:35.591401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-db62808c-fcd8-40e9-95cd-d63c0cb615d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b47624c97294f708ae129c4c277cc78', 'request_id': 'req-db62808c-fcd8-40e9-95cd-d63c0cb615d5'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:35.702211 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-db62808c-fcd8-40e9-95cd-d63c0cb615d5 admin admin] The token cache is being invalidate because project 0b47624c97294f708ae129c4c277cc78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:54:35.702712 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-db62808c-fcd8-40e9-95cd-d63c0cb615d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:35.705832 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-db62808c-fcd8-40e9-95cd-d63c0cb615d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:35.712500 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1090/2192] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:54:35 2022] DELETE /identity/v3/projects/0b47624c97294f708ae129c4c277cc78 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:54:35.717312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d792264-aebb-4937-800d-81064b687de4 service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:35.757854 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d792264-aebb-4937-800d-81064b687de4 service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:35.963815 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658d1c9-3c3e-4a92-ad97-70f1b9554a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:35.965712 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658d1c9-3c3e-4a92-ad97-70f1b9554a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:35.966266 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658d1c9-3c3e-4a92-ad97-70f1b9554a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:35.966811 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658d1c9-3c3e-4a92-ad97-70f1b9554a00 admin admin] PATH_INFO: `/v3/users/7c976366f7ea4480bf2d69025e0862a7` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:35.968806 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658d1c9-3c3e-4a92-ad97-70f1b9554a00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c976366f7ea4480bf2d69025e0862a7)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:35.972658 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658d1c9-3c3e-4a92-ad97-70f1b9554a00 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:36.225214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3658d1c9-3c3e-4a92-ad97-70f1b9554a00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c976366f7ea4480bf2d69025e0862a7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:36.392396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3658d1c9-3c3e-4a92-ad97-70f1b9554a00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c976366f7ea4480bf2d69025e0862a7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:36.392959 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3658d1c9-3c3e-4a92-ad97-70f1b9554a00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c976366f7ea4480bf2d69025e0862a7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:36.405357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3658d1c9-3c3e-4a92-ad97-70f1b9554a00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c976366f7ea4480bf2d69025e0862a7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:36.452272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3658d1c9-3c3e-4a92-ad97-70f1b9554a00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c976366f7ea4480bf2d69025e0862a7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:36.471461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1103/2193] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:54:35 2022] DELETE /identity/v3/users/7c976366f7ea4480bf2d69025e0862a7 => generated 0 bytes in 753 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:54:37.497672 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62808c-fcd8-40e9-95cd-d63c0cb615d5 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:37.499336 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-db62808c-fcd8-40e9-95cd-d63c0cb615d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:37.537693 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62808c-fcd8-40e9-95cd-d63c0cb615d5 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:37.739166 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:37.815689 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba3b55-7501-4ebc-9fe6-7531381b7ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:37.817249 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba3b55-7501-4ebc-9fe6-7531381b7ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:37.817991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba3b55-7501-4ebc-9fe6-7531381b7ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:37.819009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba3b55-7501-4ebc-9fe6-7531381b7ad3 admin admin] PATH_INFO: `/v3/projects/2dec2bce436c42d397b5c41e61538925` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:37.821808 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba3b55-7501-4ebc-9fe6-7531381b7ad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dec2bce436c42d397b5c41e61538925)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:37.843514 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba3b55-7501-4ebc-9fe6-7531381b7ad3 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:38.057234 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c9ba3b55-7501-4ebc-9fe6-7531381b7ad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dec2bce436c42d397b5c41e61538925', 'request_id': 'req-c9ba3b55-7501-4ebc-9fe6-7531381b7ad3'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:38.072953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c9ba3b55-7501-4ebc-9fe6-7531381b7ad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dec2bce436c42d397b5c41e61538925', 'request_id': 'req-c9ba3b55-7501-4ebc-9fe6-7531381b7ad3'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:38.092645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c9ba3b55-7501-4ebc-9fe6-7531381b7ad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dec2bce436c42d397b5c41e61538925', 'request_id': 'req-c9ba3b55-7501-4ebc-9fe6-7531381b7ad3'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:38.247530 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c9ba3b55-7501-4ebc-9fe6-7531381b7ad3 admin admin] The token cache is being invalidate because project 2dec2bce436c42d397b5c41e61538925 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:54:38.248281 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c9ba3b55-7501-4ebc-9fe6-7531381b7ad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:38.249537 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c9ba3b55-7501-4ebc-9fe6-7531381b7ad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:38.255609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1091/2194] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:54:37 2022] DELETE /identity/v3/projects/2dec2bce436c42d397b5c41e61538925 => generated 0 bytes in 755 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:54:38.257785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5f1fa-f08a-4685-8e82-93275659f6be None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:38.258324 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5f1fa-f08a-4685-8e82-93275659f6be None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:38.258414 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5f1fa-f08a-4685-8e82-93275659f6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:38.295275 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-abe5f1fa-f08a-4685-8e82-93275659f6be None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:38.297399 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-abe5f1fa-f08a-4685-8e82-93275659f6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:38.430543 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc04047-08a0-4751-a7b9-68f4bf72b00c None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:38.435195 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc04047-08a0-4751-a7b9-68f4bf72b00c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:38.435195 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc04047-08a0-4751-a7b9-68f4bf72b00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:38.435982 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1104/2195] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:38.468029 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-dfc04047-08a0-4751-a7b9-68f4bf72b00c None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:38.471471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-dfc04047-08a0-4751-a7b9-68f4bf72b00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:38.484705 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1092/2196] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:38.489758 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5f1fa-f08a-4685-8e82-93275659f6be None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:38.523297 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5f1fa-f08a-4685-8e82-93275659f6be None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:38.536521 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afeb732-c38c-41f0-9503-219b2084e834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:38.540408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afeb732-c38c-41f0-9503-219b2084e834 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:38.540408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afeb732-c38c-41f0-9503-219b2084e834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:38.540408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afeb732-c38c-41f0-9503-219b2084e834 admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:38.542879 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afeb732-c38c-41f0-9503-219b2084e834 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:54:38.543407 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afeb732-c38c-41f0-9503-219b2084e834 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:38.548779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afeb732-c38c-41f0-9503-219b2084e834 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:38.563893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:38.580838 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1105/2197] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:54:38.585611 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc04047-08a0-4751-a7b9-68f4bf72b00c None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:38.627343 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc04047-08a0-4751-a7b9-68f4bf72b00c None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:38.650735 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a226112-2487-4527-93f7-8dfeaba3b760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:38.653062 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a226112-2487-4527-93f7-8dfeaba3b760 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:38.655513 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a226112-2487-4527-93f7-8dfeaba3b760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:38.657063 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a226112-2487-4527-93f7-8dfeaba3b760 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:38.663012 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a226112-2487-4527-93f7-8dfeaba3b760 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:38.668636 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a226112-2487-4527-93f7-8dfeaba3b760 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:38.941612 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1093/2198] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:54:38 2022] POST /identity/v3/projects => generated 371 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:54:38.948305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afeb732-c38c-41f0-9503-219b2084e834 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:38.978225 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afeb732-c38c-41f0-9503-219b2084e834 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:39.216666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed7d44-bfea-4922-9907-c589ddb15978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:39.216666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed7d44-bfea-4922-9907-c589ddb15978 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:39.216666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed7d44-bfea-4922-9907-c589ddb15978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:39.216666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed7d44-bfea-4922-9907-c589ddb15978 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:39.217047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed7d44-bfea-4922-9907-c589ddb15978 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:39.217047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed7d44-bfea-4922-9907-c589ddb15978 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:39.348901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1106/2199] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:54:38 2022] POST /identity/v3/users => generated 286 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:54:39.350037 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a226112-2487-4527-93f7-8dfeaba3b760 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:39.366974 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a226112-2487-4527-93f7-8dfeaba3b760 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:39.382467 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e48ad0-8e2b-48b8-8110-5a2cbb6b24e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:39.383465 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e48ad0-8e2b-48b8-8110-5a2cbb6b24e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:39.384041 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e48ad0-8e2b-48b8-8110-5a2cbb6b24e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:39.384806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e48ad0-8e2b-48b8-8110-5a2cbb6b24e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:39.386055 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e48ad0-8e2b-48b8-8110-5a2cbb6b24e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:39.388332 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e48ad0-8e2b-48b8-8110-5a2cbb6b24e0 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:39.419970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1094/2200] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:54:39.426344 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed7d44-bfea-4922-9907-c589ddb15978 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:39.460716 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed7d44-bfea-4922-9907-c589ddb15978 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:39.472733 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9379bab-51c1-4f3d-9f90-8b4339f2b5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:39.475213 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9379bab-51c1-4f3d-9f90-8b4339f2b5ce admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:39.477008 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9379bab-51c1-4f3d-9f90-8b4339f2b5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:39.477577 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9379bab-51c1-4f3d-9f90-8b4339f2b5ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:39.478537 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9379bab-51c1-4f3d-9f90-8b4339f2b5ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:39.480495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9379bab-51c1-4f3d-9f90-8b4339f2b5ce admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:39.504654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e48ad0-8e2b-48b8-8110-5a2cbb6b24e0 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:39.506182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1107/2201] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:54:39.523520 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e48ad0-8e2b-48b8-8110-5a2cbb6b24e0 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:39.538305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba88f56-a9bc-40f5-95f3-4c71542d9816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:39.539726 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba88f56-a9bc-40f5-95f3-4c71542d9816 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:39.540494 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba88f56-a9bc-40f5-95f3-4c71542d9816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:39.541292 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba88f56-a9bc-40f5-95f3-4c71542d9816 admin admin] PATH_INFO: `/v3/projects/a928a25035a74178aba99df7651271bd/users/7d4819e7e269413482590bff67428b50/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:39.582350 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba88f56-a9bc-40f5-95f3-4c71542d9816 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a928a25035a74178aba99df7651271bd, user_id=7d4819e7e269413482590bff67428b50, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:39.586080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba88f56-a9bc-40f5-95f3-4c71542d9816 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:39.748631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1095/2202] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:54:39 2022] PUT /identity/v3/projects/a928a25035a74178aba99df7651271bd/users/7d4819e7e269413482590bff67428b50/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:54:39.755716 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafa75f-c6c8-4d63-9fe2-933c1f5d1f6b None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:39.756803 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafa75f-c6c8-4d63-9fe2-933c1f5d1f6b None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:39.757374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafa75f-c6c8-4d63-9fe2-933c1f5d1f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:39.795233 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-8cafa75f-c6c8-4d63-9fe2-933c1f5d1f6b None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:39.797191 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-8cafa75f-c6c8-4d63-9fe2-933c1f5d1f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:39.939887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1108/2203] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:40.051970 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba88f56-a9bc-40f5-95f3-4c71542d9816 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:40.054213 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-aba88f56-a9bc-40f5-95f3-4c71542d9816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:40.201871 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba88f56-a9bc-40f5-95f3-4c71542d9816 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:40.214695 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d2304e-f0dc-4ac1-8551-6cebf73a3f5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:40.215726 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2304e-f0dc-4ac1-8551-6cebf73a3f5e service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:40.216196 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2304e-f0dc-4ac1-8551-6cebf73a3f5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:40.216681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2304e-f0dc-4ac1-8551-6cebf73a3f5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:40.245547 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d2304e-f0dc-4ac1-8551-6cebf73a3f5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:40.263688 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d2304e-f0dc-4ac1-8551-6cebf73a3f5e service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:40.287368 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1096/2204] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:54:41.201886 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafa75f-c6c8-4d63-9fe2-933c1f5d1f6b None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:41.201886 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-8cafa75f-c6c8-4d63-9fe2-933c1f5d1f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:41.230866 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafa75f-c6c8-4d63-9fe2-933c1f5d1f6b None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:41.249723 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3fde1-3b7a-4d08-b6f8-ae2a914cbd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:41.250639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3fde1-3b7a-4d08-b6f8-ae2a914cbd3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:41.252882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3fde1-3b7a-4d08-b6f8-ae2a914cbd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:41.252882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3fde1-3b7a-4d08-b6f8-ae2a914cbd3e admin admin] PATH_INFO: `/v3/users/9d7496c6be154ac4906855191667c7ff` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:41.253800 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d3fde1-3b7a-4d08-b6f8-ae2a914cbd3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d7496c6be154ac4906855191667c7ff)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:41.257413 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d3fde1-3b7a-4d08-b6f8-ae2a914cbd3e admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:41.645830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-e9d3fde1-3b7a-4d08-b6f8-ae2a914cbd3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d7496c6be154ac4906855191667c7ff'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:41.757673 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-e9d3fde1-3b7a-4d08-b6f8-ae2a914cbd3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d7496c6be154ac4906855191667c7ff'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:41.763169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-e9d3fde1-3b7a-4d08-b6f8-ae2a914cbd3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d7496c6be154ac4906855191667c7ff'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:41.873873 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-e9d3fde1-3b7a-4d08-b6f8-ae2a914cbd3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d7496c6be154ac4906855191667c7ff'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:41.918892 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:41.957566 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-e9d3fde1-3b7a-4d08-b6f8-ae2a914cbd3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d7496c6be154ac4906855191667c7ff'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:42.025398 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1109/2205] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:54:41 2022] DELETE /identity/v3/users/9d7496c6be154ac4906855191667c7ff => generated 0 bytes in 837 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:54:42.031666 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc6e11-8a09-4060-b6bb-9308426100c1 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:42.032637 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc6e11-8a09-4060-b6bb-9308426100c1 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:42.032637 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc6e11-8a09-4060-b6bb-9308426100c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:42.123044 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:42.205993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-71cc6e11-8a09-4060-b6bb-9308426100c1 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:42.208076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-71cc6e11-8a09-4060-b6bb-9308426100c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:42.368250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1097/2206] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:54:42.491428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3fde1-3b7a-4d08-b6f8-ae2a914cbd3e admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:42.493715 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-e9d3fde1-3b7a-4d08-b6f8-ae2a914cbd3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:42.658444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3fde1-3b7a-4d08-b6f8-ae2a914cbd3e admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:42.674051 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de73b5c-1071-42af-bdea-999308538d9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:42.674910 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de73b5c-1071-42af-bdea-999308538d9d service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:42.680632 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de73b5c-1071-42af-bdea-999308538d9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:42.680632 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de73b5c-1071-42af-bdea-999308538d9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:42.700160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de73b5c-1071-42af-bdea-999308538d9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:42.707044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de73b5c-1071-42af-bdea-999308538d9d service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:42.728042 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1110/2207] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:54:43.761816 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba0446-ac47-4aa0-8cb4-9e90b70acaab None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:43.762482 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba0446-ac47-4aa0-8cb4-9e90b70acaab None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:43.762482 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba0446-ac47-4aa0-8cb4-9e90b70acaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:43.828921 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-cdba0446-ac47-4aa0-8cb4-9e90b70acaab None None] MFA Rules not processed for user `7d4819e7e269413482590bff67428b50`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:43.830964 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-cdba0446-ac47-4aa0-8cb4-9e90b70acaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:44.039924 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1098/2208] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:54:44.045658 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de73b5c-1071-42af-bdea-999308538d9d service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:44.060784 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-2de73b5c-1071-42af-bdea-999308538d9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:44.125240 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de73b5c-1071-42af-bdea-999308538d9d service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:44.167568 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f27b4-358c-4aae-898d-90d49c039edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:44.171578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f27b4-358c-4aae-898d-90d49c039edb admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:44.176614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f27b4-358c-4aae-898d-90d49c039edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:44.177101 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f27b4-358c-4aae-898d-90d49c039edb admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:44.178982 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26f27b4-358c-4aae-898d-90d49c039edb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:44.188945 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26f27b4-358c-4aae-898d-90d49c039edb admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:44.349822 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1111/2209] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:54:44 2022] POST /identity/v3/projects => generated 369 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:54:44.350286 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba0446-ac47-4aa0-8cb4-9e90b70acaab None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:44.378959 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba0446-ac47-4aa0-8cb4-9e90b70acaab None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:44.415235 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f27b4-358c-4aae-898d-90d49c039edb admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:44.420370 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f26f27b4-358c-4aae-898d-90d49c039edb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:44.491851 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8008d9-b228-406f-a536-e497ec4a0ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:44.492691 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8008d9-b228-406f-a536-e497ec4a0ea5 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:44.493090 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8008d9-b228-406f-a536-e497ec4a0ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:44.493551 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8008d9-b228-406f-a536-e497ec4a0ea5 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:44.494420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8008d9-b228-406f-a536-e497ec4a0ea5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:44.497046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8008d9-b228-406f-a536-e497ec4a0ea5 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:44.585847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f27b4-358c-4aae-898d-90d49c039edb admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:44.594745 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae009f-5808-4dfb-a52e-bbe1e3514b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:44.595541 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae009f-5808-4dfb-a52e-bbe1e3514b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:44.595824 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae009f-5808-4dfb-a52e-bbe1e3514b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:44.596108 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae009f-5808-4dfb-a52e-bbe1e3514b56 admin admin] PATH_INFO: `/v3/projects/18512864f4d942b69c471e2b7e6d77ff` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:44.598969 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae009f-5808-4dfb-a52e-bbe1e3514b56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18512864f4d942b69c471e2b7e6d77ff)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:44.599980 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae009f-5808-4dfb-a52e-bbe1e3514b56 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:44.632289 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1099/2210] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:54:44 2022] POST /identity/v3/users => generated 291 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:54:44.635984 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8008d9-b228-406f-a536-e497ec4a0ea5 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:44.663700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8008d9-b228-406f-a536-e497ec4a0ea5 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:44.675032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974dbec-e783-4176-a3aa-5436770d0a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:44.675695 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974dbec-e783-4176-a3aa-5436770d0a15 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:44.675975 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974dbec-e783-4176-a3aa-5436770d0a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:44.676245 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974dbec-e783-4176-a3aa-5436770d0a15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:44.676915 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974dbec-e783-4176-a3aa-5436770d0a15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:44.678540 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974dbec-e783-4176-a3aa-5436770d0a15 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:44.701086 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1100/2211] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:54:44.702103 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974dbec-e783-4176-a3aa-5436770d0a15 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:44.713716 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974dbec-e783-4176-a3aa-5436770d0a15 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:44.718755 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a841fd-91f6-4b1f-84a3-81746e951620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:44.720199 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a841fd-91f6-4b1f-84a3-81746e951620 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:44.720572 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a841fd-91f6-4b1f-84a3-81746e951620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:44.720861 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a841fd-91f6-4b1f-84a3-81746e951620 admin admin] PATH_INFO: `/v3/projects/8bba683094d14c5ab92b9f028b5799cb/users/d9fdb9467ab94b36ae5f84a4ef00c7cd/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:44.730684 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:44.739389 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a841fd-91f6-4b1f-84a3-81746e951620 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bba683094d14c5ab92b9f028b5799cb, user_id=d9fdb9467ab94b36ae5f84a4ef00c7cd, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:44.742487 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a841fd-91f6-4b1f-84a3-81746e951620 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:44.851025 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1101/2212] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:54:44 2022] PUT /identity/v3/projects/8bba683094d14c5ab92b9f028b5799cb/users/d9fdb9467ab94b36ae5f84a4ef00c7cd/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:54:44.851025 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a841fd-91f6-4b1f-84a3-81746e951620 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:44.947285 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-66ae009f-5808-4dfb-a52e-bbe1e3514b56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18512864f4d942b69c471e2b7e6d77ff', 'request_id': 'req-66ae009f-5808-4dfb-a52e-bbe1e3514b56'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:44.999908 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a841fd-91f6-4b1f-84a3-81746e951620 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:45.034510 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-66ae009f-5808-4dfb-a52e-bbe1e3514b56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18512864f4d942b69c471e2b7e6d77ff', 'request_id': 'req-66ae009f-5808-4dfb-a52e-bbe1e3514b56'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:45.045267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-66ae009f-5808-4dfb-a52e-bbe1e3514b56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18512864f4d942b69c471e2b7e6d77ff', 'request_id': 'req-66ae009f-5808-4dfb-a52e-bbe1e3514b56'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:45.060891 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:45.067658 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-66ae009f-5808-4dfb-a52e-bbe1e3514b56 admin admin] The token cache is being invalidate because project 18512864f4d942b69c471e2b7e6d77ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:54:45.068413 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-66ae009f-5808-4dfb-a52e-bbe1e3514b56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:45.070426 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-66ae009f-5808-4dfb-a52e-bbe1e3514b56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:54:45.078391 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1112/2213] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:54:44 2022] DELETE /identity/v3/projects/18512864f4d942b69c471e2b7e6d77ff => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:54:45.084023 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69a48a-3d0f-403e-91f0-a8e8072060d0 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:45.084023 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69a48a-3d0f-403e-91f0-a8e8072060d0 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:45.084023 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69a48a-3d0f-403e-91f0-a8e8072060d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:45.112098 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-0b69a48a-3d0f-403e-91f0-a8e8072060d0 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:45.113936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-0b69a48a-3d0f-403e-91f0-a8e8072060d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:45.161986 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce47b03-da19-49fc-a326-eabab4ab11d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:45.162660 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce47b03-da19-49fc-a326-eabab4ab11d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:45.167238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce47b03-da19-49fc-a326-eabab4ab11d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:45.167522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce47b03-da19-49fc-a326-eabab4ab11d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:45.168331 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce47b03-da19-49fc-a326-eabab4ab11d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:45.170036 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce47b03-da19-49fc-a326-eabab4ab11d7 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:45.207621 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1102/2214] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:54:45.208628 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce47b03-da19-49fc-a326-eabab4ab11d7 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:45.216659 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-cce47b03-da19-49fc-a326-eabab4ab11d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:45.236184 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1113/2215] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:45.236184 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969b82d-28da-45be-b53e-f82fa9613c33 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:45.236184 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969b82d-28da-45be-b53e-f82fa9613c33 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:45.236184 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969b82d-28da-45be-b53e-f82fa9613c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:45.266169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-7969b82d-28da-45be-b53e-f82fa9613c33 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:45.269286 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-7969b82d-28da-45be-b53e-f82fa9613c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:45.273563 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce47b03-da19-49fc-a326-eabab4ab11d7 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:45.281283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1114/2216] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:45.287882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969b82d-28da-45be-b53e-f82fa9613c33 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:45.299930 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9deff57-0ef3-4d50-9f1e-867b04b359c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:45.301364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9deff57-0ef3-4d50-9f1e-867b04b359c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:45.301364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9deff57-0ef3-4d50-9f1e-867b04b359c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:45.301364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9deff57-0ef3-4d50-9f1e-867b04b359c6 admin admin] PATH_INFO: `/v3/projects/8bba683094d14c5ab92b9f028b5799cb/users/d9fdb9467ab94b36ae5f84a4ef00c7cd/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:45.305836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969b82d-28da-45be-b53e-f82fa9613c33 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:45.307281 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9deff57-0ef3-4d50-9f1e-867b04b359c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bba683094d14c5ab92b9f028b5799cb, user_id=d9fdb9467ab94b36ae5f84a4ef00c7cd, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:45.318333 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9deff57-0ef3-4d50-9f1e-867b04b359c6 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:45.338952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea3581-b353-4c90-bfbc-b67feae35e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:45.343987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea3581-b353-4c90-bfbc-b67feae35e24 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:45.343987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea3581-b353-4c90-bfbc-b67feae35e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:45.343987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea3581-b353-4c90-bfbc-b67feae35e24 admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:45.343987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dea3581-b353-4c90-bfbc-b67feae35e24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:54:45.348880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dea3581-b353-4c90-bfbc-b67feae35e24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:45.349385 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1103/2217] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:54:45 2022] PUT /identity/v3/projects/8bba683094d14c5ab92b9f028b5799cb/users/d9fdb9467ab94b36ae5f84a4ef00c7cd/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:54:45.376540 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dea3581-b353-4c90-bfbc-b67feae35e24 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:45.408970 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1115/2218] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:54:45.410305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9deff57-0ef3-4d50-9f1e-867b04b359c6 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:45.588060 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9deff57-0ef3-4d50-9f1e-867b04b359c6 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:45.596335 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d988e5a-017f-4efe-9060-5affed8c244e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:45.596988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d988e5a-017f-4efe-9060-5affed8c244e admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:45.597265 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d988e5a-017f-4efe-9060-5affed8c244e admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:45.597537 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d988e5a-017f-4efe-9060-5affed8c244e admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:45.598183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d988e5a-017f-4efe-9060-5affed8c244e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:45.607327 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d988e5a-017f-4efe-9060-5affed8c244e admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:45.769052 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaed172-a5db-4066-899c-305fcfdd8f9f None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:45.769941 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaed172-a5db-4066-899c-305fcfdd8f9f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:45.770866 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaed172-a5db-4066-899c-305fcfdd8f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:45.796796 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1104/2219] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:54:45 2022] POST /identity/v3/projects => generated 383 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:54:45.799857 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d988e5a-017f-4efe-9060-5affed8c244e admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:45.817111 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:45.871709 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-ccaed172-a5db-4066-899c-305fcfdd8f9f None None] MFA Rules not processed for user `fdb16e7edbe143e9b96325842896f04e`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:45.878701 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ccaed172-a5db-4066-899c-305fcfdd8f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:46.053337 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d988e5a-017f-4efe-9060-5affed8c244e admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:46.075195 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c08eaa-0cc3-4de2-9ae3-dfb40102e2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:46.075944 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c08eaa-0cc3-4de2-9ae3-dfb40102e2d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:46.076301 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c08eaa-0cc3-4de2-9ae3-dfb40102e2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:46.076761 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c08eaa-0cc3-4de2-9ae3-dfb40102e2d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:46.077465 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c08eaa-0cc3-4de2-9ae3-dfb40102e2d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:46.079669 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c08eaa-0cc3-4de2-9ae3-dfb40102e2d4 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:46.170515 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1116/2220] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:46.239912 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1105/2221] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:54:45 2022] POST /identity/v3/users => generated 292 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:54:46.246908 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaed172-a5db-4066-899c-305fcfdd8f9f None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:46.269032 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaed172-a5db-4066-899c-305fcfdd8f9f None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:46.277561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89c7a9-cf3a-4bec-9507-a3964db52eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:46.283774 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89c7a9-cf3a-4bec-9507-a3964db52eff admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:46.284660 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89c7a9-cf3a-4bec-9507-a3964db52eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:46.284660 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89c7a9-cf3a-4bec-9507-a3964db52eff admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:46.285883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89c7a9-cf3a-4bec-9507-a3964db52eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:46.288072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89c7a9-cf3a-4bec-9507-a3964db52eff admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:46.321670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1117/2222] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:54:46.327780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c08eaa-0cc3-4de2-9ae3-dfb40102e2d4 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:46.368565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c08eaa-0cc3-4de2-9ae3-dfb40102e2d4 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:46.396212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a8e2a8-fe32-45ed-808c-ada95c6eb8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:46.399654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8e2a8-fe32-45ed-808c-ada95c6eb8a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:46.404109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8e2a8-fe32-45ed-808c-ada95c6eb8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:46.404109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8e2a8-fe32-45ed-808c-ada95c6eb8a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:46.408464 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a8e2a8-fe32-45ed-808c-ada95c6eb8a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:46.410471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a8e2a8-fe32-45ed-808c-ada95c6eb8a9 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:46.456496 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1106/2223] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:54:46.461403 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89c7a9-cf3a-4bec-9507-a3964db52eff admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:46.553874 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89c7a9-cf3a-4bec-9507-a3964db52eff admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:46.585068 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae5430-a56d-4924-9290-b2ae534e2b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:46.590770 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae5430-a56d-4924-9290-b2ae534e2b9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:46.590770 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae5430-a56d-4924-9290-b2ae534e2b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:46.590770 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae5430-a56d-4924-9290-b2ae534e2b9f admin admin] PATH_INFO: `/v3/projects/236c710547184e33b1b14a88402fb363/users/fd667c5354f5499686dfe858bd159aa0/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:46.637981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae5430-a56d-4924-9290-b2ae534e2b9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=236c710547184e33b1b14a88402fb363, user_id=fd667c5354f5499686dfe858bd159aa0, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:46.649452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae5430-a56d-4924-9290-b2ae534e2b9f admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:46.809642 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1118/2224] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:54:46 2022] PUT /identity/v3/projects/236c710547184e33b1b14a88402fb363/users/fd667c5354f5499686dfe858bd159aa0/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:54:46.814206 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80687304-3afd-4d39-ba91-88b25c769a47 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:46.815008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80687304-3afd-4d39-ba91-88b25c769a47 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:46.815413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80687304-3afd-4d39-ba91-88b25c769a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:46.887925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-80687304-3afd-4d39-ba91-88b25c769a47 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:46.893706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-80687304-3afd-4d39-ba91-88b25c769a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:47.133668 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1107/2225] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:54:47.191853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae5430-a56d-4924-9290-b2ae534e2b9f admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:47.194832 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-15ae5430-a56d-4924-9290-b2ae534e2b9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:47.407798 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae5430-a56d-4924-9290-b2ae534e2b9f admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:47.424201 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc99976-ab26-48b4-afc1-36310aa60ede service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:47.430842 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc99976-ab26-48b4-afc1-36310aa60ede service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:47.430842 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc99976-ab26-48b4-afc1-36310aa60ede service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:47.430842 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc99976-ab26-48b4-afc1-36310aa60ede service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:47.443226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:47.460337 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc99976-ab26-48b4-afc1-36310aa60ede service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:47.464252 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc99976-ab26-48b4-afc1-36310aa60ede service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:47.496931 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1119/2226] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:54:48.410673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18734b24-b29e-4caa-a401-e6437464d14f None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:48.411384 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18734b24-b29e-4caa-a401-e6437464d14f None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:48.411384 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18734b24-b29e-4caa-a401-e6437464d14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:48.556687 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-18734b24-b29e-4caa-a401-e6437464d14f None None] MFA Rules not processed for user `d9fdb9467ab94b36ae5f84a4ef00c7cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:48.580711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-18734b24-b29e-4caa-a401-e6437464d14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:48.894784 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1108/2227] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 485 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:48.920426 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc99976-ab26-48b4-afc1-36310aa60ede service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:48.923087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-3bc99976-ab26-48b4-afc1-36310aa60ede service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:49.064884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc99976-ab26-48b4-afc1-36310aa60ede service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:49.070626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34108e46-ac2d-4577-8a42-b3171f3d2286 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:49.071473 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34108e46-ac2d-4577-8a42-b3171f3d2286 service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:49.071855 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34108e46-ac2d-4577-8a42-b3171f3d2286 service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:49.072234 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34108e46-ac2d-4577-8a42-b3171f3d2286 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:49.074542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-34108e46-ac2d-4577-8a42-b3171f3d2286 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:49.382120 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34108e46-ac2d-4577-8a42-b3171f3d2286 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:49.390497 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34108e46-ac2d-4577-8a42-b3171f3d2286 service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:49.429169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1120/2228] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:54:53.609924 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18734b24-b29e-4caa-a401-e6437464d14f None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:53.614383 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-18734b24-b29e-4caa-a401-e6437464d14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:53.667431 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18734b24-b29e-4caa-a401-e6437464d14f None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:53.688793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452501dd-c77b-4bbb-b645-79fff9f86d4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:53.692300 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452501dd-c77b-4bbb-b645-79fff9f86d4e service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:53.694705 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452501dd-c77b-4bbb-b645-79fff9f86d4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:53.699777 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452501dd-c77b-4bbb-b645-79fff9f86d4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:53.730441 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:53.745344 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059d8f6-7e6f-4856-ba7b-786c2617a56c None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:53.745344 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059d8f6-7e6f-4856-ba7b-786c2617a56c None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:53.745344 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059d8f6-7e6f-4856-ba7b-786c2617a56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:53.748742 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452501dd-c77b-4bbb-b645-79fff9f86d4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:53.755023 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452501dd-c77b-4bbb-b645-79fff9f86d4e service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:53.849163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-c059d8f6-7e6f-4856-ba7b-786c2617a56c None None] MFA Rules not processed for user `75d793a42ec74232899b7a19e36b5867`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:53.876169 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1109/2229] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:53 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:54:53.909995 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-c059d8f6-7e6f-4856-ba7b-786c2617a56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:54.147351 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1121/2230] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:54:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:55:00.264218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452501dd-c77b-4bbb-b645-79fff9f86d4e service cinder] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:00.266386 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-452501dd-c77b-4bbb-b645-79fff9f86d4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:00.488245 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452501dd-c77b-4bbb-b645-79fff9f86d4e service cinder] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:00.511570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddf771-b817-4b41-8989-090b2ae3419d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e74dbb121d749749cf5f9d9d7221008', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:00.512241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddf771-b817-4b41-8989-090b2ae3419d service placement] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:00.512523 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddf771-b817-4b41-8989-090b2ae3419d service placement] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:00.512795 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddf771-b817-4b41-8989-090b2ae3419d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:00.526971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ddddf771-b817-4b41-8989-090b2ae3419d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:00.581282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddddf771-b817-4b41-8989-090b2ae3419d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:00.583701 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddddf771-b817-4b41-8989-090b2ae3419d service placement] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:00.623877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1110/2231] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:55:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:55:02.369665 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d973ae6-ad5b-4ddd-9841-c82f2ca8a550 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:02.379058 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d973ae6-ad5b-4ddd-9841-c82f2ca8a550 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:02.379058 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d973ae6-ad5b-4ddd-9841-c82f2ca8a550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:02.454707 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-6d973ae6-ad5b-4ddd-9841-c82f2ca8a550 None None] MFA Rules not processed for user `fd667c5354f5499686dfe858bd159aa0`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:55:02.458313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-6d973ae6-ad5b-4ddd-9841-c82f2ca8a550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:02.574605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:55:02.629230 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1122/2232] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:55:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:55:02.634142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddf771-b817-4b41-8989-090b2ae3419d service placement] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:02.682181 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddf771-b817-4b41-8989-090b2ae3419d service placement] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:02.721257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95544b82-6419-4aea-91f9-b2ff28be0385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:02.727132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95544b82-6419-4aea-91f9-b2ff28be0385 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:02.728553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95544b82-6419-4aea-91f9-b2ff28be0385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:02.733010 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95544b82-6419-4aea-91f9-b2ff28be0385 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:02.739659 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95544b82-6419-4aea-91f9-b2ff28be0385 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:02.755507 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95544b82-6419-4aea-91f9-b2ff28be0385 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:02.917787 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1111/2233] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:55:02 2022] POST /identity/v3/projects => generated 385 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:55:02.918206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d973ae6-ad5b-4ddd-9841-c82f2ca8a550 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:02.958613 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d973ae6-ad5b-4ddd-9841-c82f2ca8a550 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:03.029945 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa182ad2-d782-426f-86e6-add1ce3cee60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:03.030607 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa182ad2-d782-426f-86e6-add1ce3cee60 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:03.031451 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa182ad2-d782-426f-86e6-add1ce3cee60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:03.031926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa182ad2-d782-426f-86e6-add1ce3cee60 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:03.032889 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa182ad2-d782-426f-86e6-add1ce3cee60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:03.044854 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa182ad2-d782-426f-86e6-add1ce3cee60 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:03.179048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95544b82-6419-4aea-91f9-b2ff28be0385 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:03.181094 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1123/2234] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:55:02 2022] POST /identity/v3/users => generated 299 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:55:03.202906 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95544b82-6419-4aea-91f9-b2ff28be0385 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:03.216379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e1044-24f1-4681-8b3e-81ade350db31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:03.220560 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e1044-24f1-4681-8b3e-81ade350db31 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:03.221995 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e1044-24f1-4681-8b3e-81ade350db31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:03.222942 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e1044-24f1-4681-8b3e-81ade350db31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:03.224481 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e1044-24f1-4681-8b3e-81ade350db31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:03.228557 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e1044-24f1-4681-8b3e-81ade350db31 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:03.251753 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1112/2235] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:55:03.258646 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa182ad2-d782-426f-86e6-add1ce3cee60 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:03.279586 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa182ad2-d782-426f-86e6-add1ce3cee60 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:03.288636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a41ad-3c93-4075-b19a-3d1da1818555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:03.299216 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a41ad-3c93-4075-b19a-3d1da1818555 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:03.299216 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a41ad-3c93-4075-b19a-3d1da1818555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:03.299216 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a41ad-3c93-4075-b19a-3d1da1818555 admin admin] PATH_INFO: `/v3/projects/8616d1e7e19b40c5a4317cf56d9ada30/users/b66872cfc4634251bbc23574ad46d7b6/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:03.317604 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a41ad-3c93-4075-b19a-3d1da1818555 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8616d1e7e19b40c5a4317cf56d9ada30, user_id=b66872cfc4634251bbc23574ad46d7b6, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:03.324633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a41ad-3c93-4075-b19a-3d1da1818555 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:03.435113 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1124/2236] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:55:03 2022] PUT /identity/v3/projects/8616d1e7e19b40c5a4317cf56d9ada30/users/b66872cfc4634251bbc23574ad46d7b6/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:55:03.436148 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e1044-24f1-4681-8b3e-81ade350db31 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:03.466592 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e1044-24f1-4681-8b3e-81ade350db31 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:03.602367 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f3a62-3c43-464a-9909-52f16d62a11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:03.603797 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f3a62-3c43-464a-9909-52f16d62a11c admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:03.604478 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f3a62-3c43-464a-9909-52f16d62a11c admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:03.604804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f3a62-3c43-464a-9909-52f16d62a11c admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:03.607489 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f3a62-3c43-464a-9909-52f16d62a11c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:03.610312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f3a62-3c43-464a-9909-52f16d62a11c admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:03.630360 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1113/2237] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:55:03.637470 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a41ad-3c93-4075-b19a-3d1da1818555 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:03.655512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a41ad-3c93-4075-b19a-3d1da1818555 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:03.662605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00899d30-183f-4686-ae6b-a7d750fa1b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:03.664022 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00899d30-183f-4686-ae6b-a7d750fa1b83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:03.664628 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00899d30-183f-4686-ae6b-a7d750fa1b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:03.664912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00899d30-183f-4686-ae6b-a7d750fa1b83 admin admin] PATH_INFO: `/v3/projects/8616d1e7e19b40c5a4317cf56d9ada30/users/b66872cfc4634251bbc23574ad46d7b6/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:03.668869 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00899d30-183f-4686-ae6b-a7d750fa1b83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8616d1e7e19b40c5a4317cf56d9ada30, user_id=b66872cfc4634251bbc23574ad46d7b6, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:03.673135 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00899d30-183f-4686-ae6b-a7d750fa1b83 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:03.695769 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1125/2238] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:55:03 2022] PUT /identity/v3/projects/8616d1e7e19b40c5a4317cf56d9ada30/users/b66872cfc4634251bbc23574ad46d7b6/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:55:12.117198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61544f93-10ca-401e-a4f7-8176fec2e578 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:12.118095 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61544f93-10ca-401e-a4f7-8176fec2e578 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:12.118095 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61544f93-10ca-401e-a4f7-8176fec2e578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:12.223806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-61544f93-10ca-401e-a4f7-8176fec2e578 None None] MFA Rules not processed for user `9646662b0b3048f79ad5d2b9602f10a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:55:12.228705 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-61544f93-10ca-401e-a4f7-8176fec2e578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:12.359967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:55:12.383594 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1114/2239] 10.222.0.13 () {58 vars in 967 bytes} [Mon Aug 8 16:55:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:55:12.430853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00899d30-183f-4686-ae6b-a7d750fa1b83 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:12.434141 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-00899d30-183f-4686-ae6b-a7d750fa1b83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:12.713143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00899d30-183f-4686-ae6b-a7d750fa1b83 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:12.722197 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f2b9c-ca8f-4ed8-9379-8916fe9fb8dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c606decb832444e38a194b98f8d15f8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:12.723228 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f2b9c-ca8f-4ed8-9379-8916fe9fb8dc service swift] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:12.724108 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f2b9c-ca8f-4ed8-9379-8916fe9fb8dc service swift] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:12.728883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f2b9c-ca8f-4ed8-9379-8916fe9fb8dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:12.741885 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f2b9c-ca8f-4ed8-9379-8916fe9fb8dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:12.744361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f2b9c-ca8f-4ed8-9379-8916fe9fb8dc service swift] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:12.757874 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1126/2240] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Aug 8 16:55:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 16:55:14.497917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fa84c-0e01-4265-93f8-fc9bddba9fa2 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:14.499383 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fa84c-0e01-4265-93f8-fc9bddba9fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:14.499383 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fa84c-0e01-4265-93f8-fc9bddba9fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:14.581558 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-784fa84c-0e01-4265-93f8-fc9bddba9fa2 None None] MFA Rules not processed for user `b66872cfc4634251bbc23574ad46d7b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:55:14.583783 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-784fa84c-0e01-4265-93f8-fc9bddba9fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:14.742720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1115/2241] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:55:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:55:14.753208 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f2b9c-ca8f-4ed8-9379-8916fe9fb8dc service swift] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:14.755259 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-3b0f2b9c-ca8f-4ed8-9379-8916fe9fb8dc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:14.815241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:55:14.912199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f2b9c-ca8f-4ed8-9379-8916fe9fb8dc service swift] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:14.916628 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a244219b-b377-4544-bb4a-5372ee7515cb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea51ef02128a4b63b11049eb0daef298', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:14.917770 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244219b-b377-4544-bb4a-5372ee7515cb service glance] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:14.918277 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244219b-b377-4544-bb4a-5372ee7515cb service glance] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:14.918837 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244219b-b377-4544-bb4a-5372ee7515cb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:15.080561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a244219b-b377-4544-bb4a-5372ee7515cb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:15.083416 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a244219b-b377-4544-bb4a-5372ee7515cb service glance] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:15.118405 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1127/2242] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:55:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:55:15.762810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd1d5f-b309-4542-a016-9d32706faf30 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:15.765406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd1d5f-b309-4542-a016-9d32706faf30 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:15.766017 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd1d5f-b309-4542-a016-9d32706faf30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:15.815710 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-83fd1d5f-b309-4542-a016-9d32706faf30 None None] MFA Rules not processed for user `9646662b0b3048f79ad5d2b9602f10a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:55:15.818287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-83fd1d5f-b309-4542-a016-9d32706faf30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:15.847929 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1116/2243] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Aug 8 16:55:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:55:15.877384 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a244219b-b377-4544-bb4a-5372ee7515cb service glance] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:15.910077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a244219b-b377-4544-bb4a-5372ee7515cb service glance] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:15.922258 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c32b7f-a9c6-4c5b-8f5d-323abab4f8f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c606decb832444e38a194b98f8d15f8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:15.924214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c32b7f-a9c6-4c5b-8f5d-323abab4f8f9 service swift] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:15.925135 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c32b7f-a9c6-4c5b-8f5d-323abab4f8f9 service swift] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:15.929140 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c32b7f-a9c6-4c5b-8f5d-323abab4f8f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:15.971640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c32b7f-a9c6-4c5b-8f5d-323abab4f8f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:15.977778 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c32b7f-a9c6-4c5b-8f5d-323abab4f8f9 service swift] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:16.012150 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1128/2244] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Aug 8 16:55:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 16:55:18.601491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd1d5f-b309-4542-a016-9d32706faf30 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:18.604148 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-83fd1d5f-b309-4542-a016-9d32706faf30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:18.967445 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd1d5f-b309-4542-a016-9d32706faf30 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:18.991908 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ddedb-51eb-4201-872e-2b5747a4a18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:18.993732 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ddedb-51eb-4201-872e-2b5747a4a18a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:18.994520 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ddedb-51eb-4201-872e-2b5747a4a18a admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:18.995316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ddedb-51eb-4201-872e-2b5747a4a18a admin admin] PATH_INFO: `/v3/users/00fdea0388e34b1eb5bfff49a3de5b57` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:19.004832 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ddedb-51eb-4201-872e-2b5747a4a18a admin admin] RBAC: Authorizing `identity:delete_user(user_id=00fdea0388e34b1eb5bfff49a3de5b57)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:19.009234 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ddedb-51eb-4201-872e-2b5747a4a18a admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:19.359656 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-1a4ddedb-51eb-4201-872e-2b5747a4a18a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00fdea0388e34b1eb5bfff49a3de5b57'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:19.469466 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-1a4ddedb-51eb-4201-872e-2b5747a4a18a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00fdea0388e34b1eb5bfff49a3de5b57'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:19.471328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-1a4ddedb-51eb-4201-872e-2b5747a4a18a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00fdea0388e34b1eb5bfff49a3de5b57'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:19.496880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-1a4ddedb-51eb-4201-872e-2b5747a4a18a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00fdea0388e34b1eb5bfff49a3de5b57'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:19.498518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-1a4ddedb-51eb-4201-872e-2b5747a4a18a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00fdea0388e34b1eb5bfff49a3de5b57'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:19.533039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:55:19.567358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1117/2245] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:55:18 2022] DELETE /identity/v3/users/00fdea0388e34b1eb5bfff49a3de5b57 => generated 0 bytes in 966 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:55:20.068263 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cb8bc-1fb6-4f9b-90f9-45ec94fda648 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:20.069177 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cb8bc-1fb6-4f9b-90f9-45ec94fda648 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:20.069643 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cb8bc-1fb6-4f9b-90f9-45ec94fda648 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:20.071874 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1129/2246] 10.222.0.13 () {54 vars in 921 bytes} [Mon Aug 8 16:55:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:55:20.390177 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d488293-6c0e-4162-ba73-44ea66c98f48 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:20.391544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d488293-6c0e-4162-ba73-44ea66c98f48 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:20.392274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d488293-6c0e-4162-ba73-44ea66c98f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:20.457741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-8d488293-6c0e-4162-ba73-44ea66c98f48 None None] MFA Rules not processed for user `6e74dbb121d749749cf5f9d9d7221008`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:55:20.464430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-8d488293-6c0e-4162-ba73-44ea66c98f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:20.589350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cb8bc-1fb6-4f9b-90f9-45ec94fda648 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:20.591264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ae0cb8bc-1fb6-4f9b-90f9-45ec94fda648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:20.753917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1118/2247] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Aug 8 16:55:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:55:20.771965 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d488293-6c0e-4162-ba73-44ea66c98f48 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:20.773698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-8d488293-6c0e-4162-ba73-44ea66c98f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:20.806894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d488293-6c0e-4162-ba73-44ea66c98f48 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:20.834889 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cb8bc-1fb6-4f9b-90f9-45ec94fda648 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:20.836314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dea3c5-9d10-4bf2-8a45-187255614aa9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e74dbb121d749749cf5f9d9d7221008', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:20.836979 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dea3c5-9d10-4bf2-8a45-187255614aa9 service placement] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:20.837260 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dea3c5-9d10-4bf2-8a45-187255614aa9 service placement] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:20.837531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dea3c5-9d10-4bf2-8a45-187255614aa9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:20.847921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71744b-0d4b-452d-b0d8-8a71273cdc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:20.849634 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71744b-0d4b-452d-b0d8-8a71273cdc91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:20.850460 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71744b-0d4b-452d-b0d8-8a71273cdc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:20.851165 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71744b-0d4b-452d-b0d8-8a71273cdc91 admin admin] PATH_INFO: `/v3/users/7d4819e7e269413482590bff67428b50` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:20.853722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e71744b-0d4b-452d-b0d8-8a71273cdc91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d4819e7e269413482590bff67428b50)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:20.856948 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e71744b-0d4b-452d-b0d8-8a71273cdc91 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:20.888338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dea3c5-9d10-4bf2-8a45-187255614aa9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:20.899703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dea3c5-9d10-4bf2-8a45-187255614aa9 service placement] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:20.982416 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1119/2248] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:55:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:55:20.984623 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dea3c5-9d10-4bf2-8a45-187255614aa9 service placement] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:20.986882 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-08dea3c5-9d10-4bf2-8a45-187255614aa9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:21.007129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dea3c5-9d10-4bf2-8a45-187255614aa9 service placement] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:21.022119 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790333b0-72ae-444a-9a16-16e40850609d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:21.022119 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790333b0-72ae-444a-9a16-16e40850609d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:21.022119 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790333b0-72ae-444a-9a16-16e40850609d admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:21.022119 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790333b0-72ae-444a-9a16-16e40850609d admin admin] PATH_INFO: `/v3/projects/ec57970389084368ac985ecb58645bd1` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:21.023335 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790333b0-72ae-444a-9a16-16e40850609d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec57970389084368ac985ecb58645bd1)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:21.025861 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790333b0-72ae-444a-9a16-16e40850609d admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:21.237915 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-790333b0-72ae-444a-9a16-16e40850609d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec57970389084368ac985ecb58645bd1', 'request_id': 'req-790333b0-72ae-444a-9a16-16e40850609d'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:21.253909 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-790333b0-72ae-444a-9a16-16e40850609d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec57970389084368ac985ecb58645bd1', 'request_id': 'req-790333b0-72ae-444a-9a16-16e40850609d'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:21.273368 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-790333b0-72ae-444a-9a16-16e40850609d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec57970389084368ac985ecb58645bd1', 'request_id': 'req-790333b0-72ae-444a-9a16-16e40850609d'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:21.303698 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-4e71744b-0d4b-452d-b0d8-8a71273cdc91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d4819e7e269413482590bff67428b50'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:21.323693 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:55:21.368385 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-790333b0-72ae-444a-9a16-16e40850609d admin admin] The token cache is being invalidate because project ec57970389084368ac985ecb58645bd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:55:21.368764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-790333b0-72ae-444a-9a16-16e40850609d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:21.369551 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-790333b0-72ae-444a-9a16-16e40850609d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:21.376960 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1120/2249] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:55:20 2022] DELETE /identity/v3/projects/ec57970389084368ac985ecb58645bd1 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:55:21.378536 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f63b06-8127-4f8f-a8ee-32aae098f4c3 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:21.383602 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f63b06-8127-4f8f-a8ee-32aae098f4c3 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:21.383602 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f63b06-8127-4f8f-a8ee-32aae098f4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:21.428921 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-48f63b06-8127-4f8f-a8ee-32aae098f4c3 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:55:21.430196 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-48f63b06-8127-4f8f-a8ee-32aae098f4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:21.463745 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-4e71744b-0d4b-452d-b0d8-8a71273cdc91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d4819e7e269413482590bff67428b50'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:21.465033 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-4e71744b-0d4b-452d-b0d8-8a71273cdc91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d4819e7e269413482590bff67428b50'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:21.500124 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-4e71744b-0d4b-452d-b0d8-8a71273cdc91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d4819e7e269413482590bff67428b50'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:21.582354 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-4e71744b-0d4b-452d-b0d8-8a71273cdc91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d4819e7e269413482590bff67428b50'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:21.619489 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1130/2250] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:55:20 2022] DELETE /identity/v3/users/7d4819e7e269413482590bff67428b50 => generated 0 bytes in 1026 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:55:21.627566 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:55:21.660231 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1121/2251] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:55:21.660561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd01cc-a958-4854-98ac-1a2da6666ee5 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:21.660887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd01cc-a958-4854-98ac-1a2da6666ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:21.661360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd01cc-a958-4854-98ac-1a2da6666ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:21.731562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-61cd01cc-a958-4854-98ac-1a2da6666ee5 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:55:21.733922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-61cd01cc-a958-4854-98ac-1a2da6666ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:21.911807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f63b06-8127-4f8f-a8ee-32aae098f4c3 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:21.922692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1131/2252] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:55:21.928774 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f63b06-8127-4f8f-a8ee-32aae098f4c3 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:21.953469 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23de7b-c136-451f-882c-dfc6e98f2aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:21.954218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23de7b-c136-451f-882c-dfc6e98f2aed admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:21.956769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23de7b-c136-451f-882c-dfc6e98f2aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:21.957174 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23de7b-c136-451f-882c-dfc6e98f2aed admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:21.958393 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23de7b-c136-451f-882c-dfc6e98f2aed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:55:21.958393 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23de7b-c136-451f-882c-dfc6e98f2aed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:21.961659 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23de7b-c136-451f-882c-dfc6e98f2aed admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:22.016326 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd01cc-a958-4854-98ac-1a2da6666ee5 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:22.029491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1122/2253] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:55:22.099027 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd01cc-a958-4854-98ac-1a2da6666ee5 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:22.142260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da15a4-2c42-4895-8118-31b7a3bf2c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:22.145608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da15a4-2c42-4895-8118-31b7a3bf2c9b admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:22.146061 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da15a4-2c42-4895-8118-31b7a3bf2c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:22.146472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da15a4-2c42-4895-8118-31b7a3bf2c9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:22.150453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da15a4-2c42-4895-8118-31b7a3bf2c9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:22.150453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da15a4-2c42-4895-8118-31b7a3bf2c9b admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:22.352168 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23de7b-c136-451f-882c-dfc6e98f2aed admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:22.353392 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1132/2254] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:55:22 2022] POST /identity/v3/projects => generated 369 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:55:22.368876 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23de7b-c136-451f-882c-dfc6e98f2aed admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:22.518846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59894ce0-2afe-4bb8-8bb7-60f1140f47e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:22.520491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59894ce0-2afe-4bb8-8bb7-60f1140f47e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:22.521037 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59894ce0-2afe-4bb8-8bb7-60f1140f47e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:22.522263 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59894ce0-2afe-4bb8-8bb7-60f1140f47e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:22.523207 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59894ce0-2afe-4bb8-8bb7-60f1140f47e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:22.525720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59894ce0-2afe-4bb8-8bb7-60f1140f47e5 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:22.677220 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1123/2255] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:55:22 2022] POST /identity/v3/users => generated 285 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:55:22.678028 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da15a4-2c42-4895-8118-31b7a3bf2c9b admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:22.719501 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da15a4-2c42-4895-8118-31b7a3bf2c9b admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:22.745057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f4215-1d36-4dfc-af55-3d8328578be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:22.746690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f4215-1d36-4dfc-af55-3d8328578be4 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:22.748448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f4215-1d36-4dfc-af55-3d8328578be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:22.748448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f4215-1d36-4dfc-af55-3d8328578be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:22.760005 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f4215-1d36-4dfc-af55-3d8328578be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:22.763675 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f4215-1d36-4dfc-af55-3d8328578be4 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:22.808647 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1133/2256] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:55:22.813352 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59894ce0-2afe-4bb8-8bb7-60f1140f47e5 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:22.845998 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59894ce0-2afe-4bb8-8bb7-60f1140f47e5 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:22.894192 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827f24c-a339-4ad6-819b-12e7e14b6cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:22.905839 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827f24c-a339-4ad6-819b-12e7e14b6cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:22.906713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827f24c-a339-4ad6-819b-12e7e14b6cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:22.907075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827f24c-a339-4ad6-819b-12e7e14b6cb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:22.909329 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8827f24c-a339-4ad6-819b-12e7e14b6cb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:22.911653 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8827f24c-a339-4ad6-819b-12e7e14b6cb3 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:22.968067 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f4215-1d36-4dfc-af55-3d8328578be4 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:22.973010 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1124/2257] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:55:23.002388 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f4215-1d36-4dfc-af55-3d8328578be4 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:23.035115 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667bebc7-d318-4f78-8d0a-0de6fce30719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:23.045788 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bebc7-d318-4f78-8d0a-0de6fce30719 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:23.045788 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bebc7-d318-4f78-8d0a-0de6fce30719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:23.045788 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bebc7-d318-4f78-8d0a-0de6fce30719 admin admin] PATH_INFO: `/v3/projects/1496f38f920d4bb995363e08ecbab3e7/users/aeb73c02c74046b5b0b85f8cb588455a/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:23.078056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827f24c-a339-4ad6-819b-12e7e14b6cb3 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:23.084775 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-8827f24c-a339-4ad6-819b-12e7e14b6cb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:23.141504 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827f24c-a339-4ad6-819b-12e7e14b6cb3 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:23.163318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a63273-6a35-41d1-9901-2984a86904a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:23.164336 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a63273-6a35-41d1-9901-2984a86904a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:23.164768 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a63273-6a35-41d1-9901-2984a86904a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:23.165324 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a63273-6a35-41d1-9901-2984a86904a1 admin admin] PATH_INFO: `/v3/projects/a928a25035a74178aba99df7651271bd` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:23.168486 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a63273-6a35-41d1-9901-2984a86904a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a928a25035a74178aba99df7651271bd)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:23.188683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a63273-6a35-41d1-9901-2984a86904a1 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:23.194266 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667bebc7-d318-4f78-8d0a-0de6fce30719 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1496f38f920d4bb995363e08ecbab3e7, user_id=aeb73c02c74046b5b0b85f8cb588455a, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:23.203889 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667bebc7-d318-4f78-8d0a-0de6fce30719 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:23.332487 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1134/2258] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:55:22 2022] PUT /identity/v3/projects/1496f38f920d4bb995363e08ecbab3e7/users/aeb73c02c74046b5b0b85f8cb588455a/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:55:23.349118 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e678f-2db8-4760-ad2e-101dd9ea2c5f None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:23.349584 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e678f-2db8-4760-ad2e-101dd9ea2c5f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:23.350034 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e678f-2db8-4760-ad2e-101dd9ea2c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:23.527744 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-449e678f-2db8-4760-ad2e-101dd9ea2c5f None None] MFA Rules not processed for user `aeb73c02c74046b5b0b85f8cb588455a`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:55:23.529990 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-449e678f-2db8-4760-ad2e-101dd9ea2c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:23.554483 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-15a63273-6a35-41d1-9901-2984a86904a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a928a25035a74178aba99df7651271bd', 'request_id': 'req-15a63273-6a35-41d1-9901-2984a86904a1'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:23.586204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-15a63273-6a35-41d1-9901-2984a86904a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a928a25035a74178aba99df7651271bd', 'request_id': 'req-15a63273-6a35-41d1-9901-2984a86904a1'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:23.603489 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-15a63273-6a35-41d1-9901-2984a86904a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a928a25035a74178aba99df7651271bd', 'request_id': 'req-15a63273-6a35-41d1-9901-2984a86904a1'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:23.736638 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-15a63273-6a35-41d1-9901-2984a86904a1 admin admin] The token cache is being invalidate because project a928a25035a74178aba99df7651271bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:55:23.737169 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-15a63273-6a35-41d1-9901-2984a86904a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:23.740456 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-15a63273-6a35-41d1-9901-2984a86904a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:23.743661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1125/2259] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:55:23 2022] DELETE /identity/v3/projects/a928a25035a74178aba99df7651271bd => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:55:23.750451 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a63273-6a35-41d1-9901-2984a86904a1 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:23.757715 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-15a63273-6a35-41d1-9901-2984a86904a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:23.879318 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:55:23.901408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1135/2260] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:55:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 554 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:55:23.930494 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449e678f-2db8-4760-ad2e-101dd9ea2c5f None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:23.934475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-449e678f-2db8-4760-ad2e-101dd9ea2c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:24.069426 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:55:24.115419 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a63273-6a35-41d1-9901-2984a86904a1 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:24.143977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89364e5f-66bc-4b17-8e4b-9cb7ff7f485c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:24.146037 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89364e5f-66bc-4b17-8e4b-9cb7ff7f485c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:24.146546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89364e5f-66bc-4b17-8e4b-9cb7ff7f485c admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:24.147107 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89364e5f-66bc-4b17-8e4b-9cb7ff7f485c admin admin] PATH_INFO: `/v3/users/d9fdb9467ab94b36ae5f84a4ef00c7cd` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:24.168015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89364e5f-66bc-4b17-8e4b-9cb7ff7f485c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9fdb9467ab94b36ae5f84a4ef00c7cd)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:24.179532 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89364e5f-66bc-4b17-8e4b-9cb7ff7f485c admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:24.192478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449e678f-2db8-4760-ad2e-101dd9ea2c5f None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:24.208720 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657875d-6569-44b5-abd9-b24a0d7b33a6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea51ef02128a4b63b11049eb0daef298', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:24.215070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657875d-6569-44b5-abd9-b24a0d7b33a6 service glance] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:24.217686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657875d-6569-44b5-abd9-b24a0d7b33a6 service glance] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:24.218142 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657875d-6569-44b5-abd9-b24a0d7b33a6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:24.256898 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9657875d-6569-44b5-abd9-b24a0d7b33a6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:24.261150 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9657875d-6569-44b5-abd9-b24a0d7b33a6 service glance] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:24.291553 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1136/2261] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:55:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:55:24.537159 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-89364e5f-66bc-4b17-8e4b-9cb7ff7f485c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9fdb9467ab94b36ae5f84a4ef00c7cd'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:24.738774 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-89364e5f-66bc-4b17-8e4b-9cb7ff7f485c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9fdb9467ab94b36ae5f84a4ef00c7cd'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:24.740471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-89364e5f-66bc-4b17-8e4b-9cb7ff7f485c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9fdb9467ab94b36ae5f84a4ef00c7cd'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:24.763968 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-89364e5f-66bc-4b17-8e4b-9cb7ff7f485c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9fdb9467ab94b36ae5f84a4ef00c7cd'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:24.767129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-89364e5f-66bc-4b17-8e4b-9cb7ff7f485c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9fdb9467ab94b36ae5f84a4ef00c7cd'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:24.888156 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1126/2262] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:55:23 2022] DELETE /identity/v3/users/d9fdb9467ab94b36ae5f84a4ef00c7cd => generated 0 bytes in 1139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:55:25.530930 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657875d-6569-44b5-abd9-b24a0d7b33a6 service glance] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:25.541617 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657875d-6569-44b5-abd9-b24a0d7b33a6 service glance] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:25.671333 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12e1f5-322d-4671-ba76-5c37f7680ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:25.671540 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12e1f5-322d-4671-ba76-5c37f7680ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:25.671709 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12e1f5-322d-4671-ba76-5c37f7680ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:25.671974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12e1f5-322d-4671-ba76-5c37f7680ad3 admin admin] PATH_INFO: `/v3/projects/8bba683094d14c5ab92b9f028b5799cb` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:25.673305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12e1f5-322d-4671-ba76-5c37f7680ad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bba683094d14c5ab92b9f028b5799cb)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:25.675298 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12e1f5-322d-4671-ba76-5c37f7680ad3 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:25.857603 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-6e12e1f5-322d-4671-ba76-5c37f7680ad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bba683094d14c5ab92b9f028b5799cb', 'request_id': 'req-6e12e1f5-322d-4671-ba76-5c37f7680ad3'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:25.984469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-6e12e1f5-322d-4671-ba76-5c37f7680ad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bba683094d14c5ab92b9f028b5799cb', 'request_id': 'req-6e12e1f5-322d-4671-ba76-5c37f7680ad3'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:26.012437 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-6e12e1f5-322d-4671-ba76-5c37f7680ad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bba683094d14c5ab92b9f028b5799cb', 'request_id': 'req-6e12e1f5-322d-4671-ba76-5c37f7680ad3'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:26.060266 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-6e12e1f5-322d-4671-ba76-5c37f7680ad3 admin admin] The token cache is being invalidate because project 8bba683094d14c5ab92b9f028b5799cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:55:26.067662 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-6e12e1f5-322d-4671-ba76-5c37f7680ad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:26.067662 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-6e12e1f5-322d-4671-ba76-5c37f7680ad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:26.084593 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a04376-8c2b-47e3-86ab-bea87fc2139d None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:26.086007 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a04376-8c2b-47e3-86ab-bea87fc2139d None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:26.086588 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a04376-8c2b-47e3-86ab-bea87fc2139d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:26.090694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1137/2263] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:55:25 2022] DELETE /identity/v3/projects/8bba683094d14c5ab92b9f028b5799cb => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:55:26.135279 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-91a04376-8c2b-47e3-86ab-bea87fc2139d None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:55:26.137059 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-91a04376-8c2b-47e3-86ab-bea87fc2139d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:26.298171 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d77c7-655d-4c8e-9990-4a57a0622968 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:26.298790 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d77c7-655d-4c8e-9990-4a57a0622968 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:26.299254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d77c7-655d-4c8e-9990-4a57a0622968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:26.303832 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1127/2264] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:55:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:55:26.381736 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-4e3d77c7-655d-4c8e-9990-4a57a0622968 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:55:26.383551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-4e3d77c7-655d-4c8e-9990-4a57a0622968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:26.414318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a04376-8c2b-47e3-86ab-bea87fc2139d None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:26.427926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1138/2265] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:55:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:55:26.434391 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a04376-8c2b-47e3-86ab-bea87fc2139d None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:26.451080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f88564-e456-4f89-99dc-64e399ccee32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:26.452528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f88564-e456-4f89-99dc-64e399ccee32 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:26.453044 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f88564-e456-4f89-99dc-64e399ccee32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:26.453359 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f88564-e456-4f89-99dc-64e399ccee32 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:26.454220 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f88564-e456-4f89-99dc-64e399ccee32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:55:26.454990 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f88564-e456-4f89-99dc-64e399ccee32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:26.459210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f88564-e456-4f89-99dc-64e399ccee32 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:26.495715 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1128/2266] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:55:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:55:26.507592 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d77c7-655d-4c8e-9990-4a57a0622968 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:26.526549 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d77c7-655d-4c8e-9990-4a57a0622968 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:26.548919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9893687-db0d-494e-841e-b9693a04b4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:26.550406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9893687-db0d-494e-841e-b9693a04b4b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:26.550822 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9893687-db0d-494e-841e-b9693a04b4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:26.553342 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9893687-db0d-494e-841e-b9693a04b4b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:26.554142 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9893687-db0d-494e-841e-b9693a04b4b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:26.560223 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9893687-db0d-494e-841e-b9693a04b4b1 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:26.742793 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1139/2267] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:55:26 2022] POST /identity/v3/projects => generated 375 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:55:26.754414 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f88564-e456-4f89-99dc-64e399ccee32 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:26.795067 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f88564-e456-4f89-99dc-64e399ccee32 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:26.968490 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:55:26.984689 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c386fac-818d-4bf9-a4cb-5ed21f1ddebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:26.985498 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c386fac-818d-4bf9-a4cb-5ed21f1ddebf admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:26.985832 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c386fac-818d-4bf9-a4cb-5ed21f1ddebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:26.986105 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c386fac-818d-4bf9-a4cb-5ed21f1ddebf admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:26.986783 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c386fac-818d-4bf9-a4cb-5ed21f1ddebf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:26.988983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c386fac-818d-4bf9-a4cb-5ed21f1ddebf admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:27.120533 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1129/2268] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:55:26 2022] POST /identity/v3/users => generated 288 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:55:27.122478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9893687-db0d-494e-841e-b9693a04b4b1 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:27.142361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:55:27.164051 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9893687-db0d-494e-841e-b9693a04b4b1 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:27.173460 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8e1e8-229c-43cf-ac97-56f11d77c7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:27.174911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8e1e8-229c-43cf-ac97-56f11d77c7a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:27.175623 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8e1e8-229c-43cf-ac97-56f11d77c7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:27.176275 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8e1e8-229c-43cf-ac97-56f11d77c7a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:27.178303 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8e1e8-229c-43cf-ac97-56f11d77c7a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:27.185384 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8e1e8-229c-43cf-ac97-56f11d77c7a7 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:27.218898 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c386fac-818d-4bf9-a4cb-5ed21f1ddebf admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:27.219121 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1140/2269] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:55:27.243297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c386fac-818d-4bf9-a4cb-5ed21f1ddebf admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:27.254559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fdb1c-4c23-4e81-9851-b5fcb19c742f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:27.255763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fdb1c-4c23-4e81-9851-b5fcb19c742f admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:27.256274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fdb1c-4c23-4e81-9851-b5fcb19c742f admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:27.256816 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fdb1c-4c23-4e81-9851-b5fcb19c742f admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:27.258090 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fdb1c-4c23-4e81-9851-b5fcb19c742f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:27.261635 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fdb1c-4c23-4e81-9851-b5fcb19c742f admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:27.288763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1130/2270] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:55:27.295657 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8e1e8-229c-43cf-ac97-56f11d77c7a7 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:27.319783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8e1e8-229c-43cf-ac97-56f11d77c7a7 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:27.328817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4f7ec-bb9e-437a-889c-89ebcbccbfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:27.330688 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4f7ec-bb9e-437a-889c-89ebcbccbfb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:27.330688 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4f7ec-bb9e-437a-889c-89ebcbccbfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:27.331136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4f7ec-bb9e-437a-889c-89ebcbccbfb9 admin admin] PATH_INFO: `/v3/projects/169aee32c78844128b040c9a615f3432/users/cb2e76b64d3042a9ab8fae2d59f984a5/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:27.369951 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4f7ec-bb9e-437a-889c-89ebcbccbfb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=169aee32c78844128b040c9a615f3432, user_id=cb2e76b64d3042a9ab8fae2d59f984a5, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:27.376520 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4f7ec-bb9e-437a-889c-89ebcbccbfb9 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:27.521973 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1141/2271] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:55:27 2022] PUT /identity/v3/projects/169aee32c78844128b040c9a615f3432/users/cb2e76b64d3042a9ab8fae2d59f984a5/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:55:27.523110 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fdf66-52c3-4469-b6fe-018ddf57daba None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:27.523617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fdf66-52c3-4469-b6fe-018ddf57daba None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:27.524119 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fdf66-52c3-4469-b6fe-018ddf57daba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:27.604616 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-8f4fdf66-52c3-4469-b6fe-018ddf57daba None None] MFA Rules not processed for user `cb2e76b64d3042a9ab8fae2d59f984a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:55:27.606770 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-8f4fdf66-52c3-4469-b6fe-018ddf57daba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:27.791606 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1131/2272] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:55:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:55:27.809595 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4f7ec-bb9e-437a-889c-89ebcbccbfb9 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:27.812590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-69c4f7ec-bb9e-437a-889c-89ebcbccbfb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:27.917116 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4f7ec-bb9e-437a-889c-89ebcbccbfb9 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:27.929448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8ef48-683a-4b1f-be4d-ed3f88e62e0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:27.932367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8ef48-683a-4b1f-be4d-ed3f88e62e0d service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:27.932807 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8ef48-683a-4b1f-be4d-ed3f88e62e0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:27.933191 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8ef48-683a-4b1f-be4d-ed3f88e62e0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:27.973927 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8ef48-683a-4b1f-be4d-ed3f88e62e0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:27.978824 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8ef48-683a-4b1f-be4d-ed3f88e62e0d service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:28.022138 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1142/2273] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:55:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:55:33.689860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fdf66-52c3-4469-b6fe-018ddf57daba None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:33.844405 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fdf66-52c3-4469-b6fe-018ddf57daba None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:33.852541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd6f5f-9bd7-4f50-9c6a-87349a730218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:33.853598 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd6f5f-9bd7-4f50-9c6a-87349a730218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:33.854115 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd6f5f-9bd7-4f50-9c6a-87349a730218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:33.854554 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd6f5f-9bd7-4f50-9c6a-87349a730218 admin admin] PATH_INFO: `/v3/users/cb2e76b64d3042a9ab8fae2d59f984a5` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:33.861427 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd6f5f-9bd7-4f50-9c6a-87349a730218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb2e76b64d3042a9ab8fae2d59f984a5)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:33.861810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd6f5f-9bd7-4f50-9c6a-87349a730218 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:34.133382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-01cd6f5f-9bd7-4f50-9c6a-87349a730218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb2e76b64d3042a9ab8fae2d59f984a5'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:34.223984 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-01cd6f5f-9bd7-4f50-9c6a-87349a730218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb2e76b64d3042a9ab8fae2d59f984a5'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:34.225849 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-01cd6f5f-9bd7-4f50-9c6a-87349a730218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb2e76b64d3042a9ab8fae2d59f984a5'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:34.247866 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-01cd6f5f-9bd7-4f50-9c6a-87349a730218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb2e76b64d3042a9ab8fae2d59f984a5'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:34.249891 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-01cd6f5f-9bd7-4f50-9c6a-87349a730218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb2e76b64d3042a9ab8fae2d59f984a5'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:34.263397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:55:34.311003 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1132/2274] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:55:33 2022] DELETE /identity/v3/users/cb2e76b64d3042a9ab8fae2d59f984a5 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:55:34.311293 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f85cd-1dfe-4b96-9b0a-533028c603d0 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:34.311293 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f85cd-1dfe-4b96-9b0a-533028c603d0 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:34.311293 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f85cd-1dfe-4b96-9b0a-533028c603d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:34.352716 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-986f85cd-1dfe-4b96-9b0a-533028c603d0 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:55:34.355078 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-986f85cd-1dfe-4b96-9b0a-533028c603d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:34.481655 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1143/2275] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:55:34.494279 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd6f5f-9bd7-4f50-9c6a-87349a730218 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:34.495831 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-01cd6f5f-9bd7-4f50-9c6a-87349a730218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:34.571211 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd6f5f-9bd7-4f50-9c6a-87349a730218 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:34.575849 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe084f3-6570-4b58-98d0-38c791f9dc83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:34.576679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe084f3-6570-4b58-98d0-38c791f9dc83 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:34.577053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe084f3-6570-4b58-98d0-38c791f9dc83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:34.577638 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe084f3-6570-4b58-98d0-38c791f9dc83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:34.588025 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe084f3-6570-4b58-98d0-38c791f9dc83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:34.589497 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe084f3-6570-4b58-98d0-38c791f9dc83 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:34.601875 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1133/2276] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:55:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:55:35.238627 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f85cd-1dfe-4b96-9b0a-533028c603d0 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:35.243821 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-986f85cd-1dfe-4b96-9b0a-533028c603d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:35.271404 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f85cd-1dfe-4b96-9b0a-533028c603d0 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:35.277175 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f225b2-c404-42cf-b1ea-4f19e04ebcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:35.281938 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f225b2-c404-42cf-b1ea-4f19e04ebcca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:35.281938 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f225b2-c404-42cf-b1ea-4f19e04ebcca admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:35.281938 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f225b2-c404-42cf-b1ea-4f19e04ebcca admin admin] PATH_INFO: `/v3/projects/169aee32c78844128b040c9a615f3432` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:35.281938 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f225b2-c404-42cf-b1ea-4f19e04ebcca admin admin] RBAC: Authorizing `identity:delete_project(project_id=169aee32c78844128b040c9a615f3432)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:35.283243 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f225b2-c404-42cf-b1ea-4f19e04ebcca admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:35.470100 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-e2f225b2-c404-42cf-b1ea-4f19e04ebcca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '169aee32c78844128b040c9a615f3432', 'request_id': 'req-e2f225b2-c404-42cf-b1ea-4f19e04ebcca'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:35.612308 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-e2f225b2-c404-42cf-b1ea-4f19e04ebcca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '169aee32c78844128b040c9a615f3432', 'request_id': 'req-e2f225b2-c404-42cf-b1ea-4f19e04ebcca'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:35.629080 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-e2f225b2-c404-42cf-b1ea-4f19e04ebcca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '169aee32c78844128b040c9a615f3432', 'request_id': 'req-e2f225b2-c404-42cf-b1ea-4f19e04ebcca'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:35.637645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:55:35.653376 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-e2f225b2-c404-42cf-b1ea-4f19e04ebcca admin admin] The token cache is being invalidate because project 169aee32c78844128b040c9a615f3432 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:55:35.656195 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-e2f225b2-c404-42cf-b1ea-4f19e04ebcca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:35.657360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-e2f225b2-c404-42cf-b1ea-4f19e04ebcca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:55:35.659388 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1144/2277] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:55:35 2022] DELETE /identity/v3/projects/169aee32c78844128b040c9a615f3432 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:55:36.731462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edae0e-b483-40e1-8a2f-8caeafba169c None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:36.732029 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edae0e-b483-40e1-8a2f-8caeafba169c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:36.732029 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edae0e-b483-40e1-8a2f-8caeafba169c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:36.759316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-90edae0e-b483-40e1-8a2f-8caeafba169c None None] MFA Rules not processed for user `fd667c5354f5499686dfe858bd159aa0`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:55:36.761953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-90edae0e-b483-40e1-8a2f-8caeafba169c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:36.837286 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1134/2278] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:55:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:55:45.796165 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:DELETE=1,keystone:SELECT=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:55:46.101884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5841f11-381c-4fb6-81e7-b0b2a60f8309 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:46.102334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5841f11-381c-4fb6-81e7-b0b2a60f8309 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:46.102643 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5841f11-381c-4fb6-81e7-b0b2a60f8309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:46.126323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-f5841f11-381c-4fb6-81e7-b0b2a60f8309 None None] MFA Rules not processed for user `9646662b0b3048f79ad5d2b9602f10a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:55:46.128191 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f5841f11-381c-4fb6-81e7-b0b2a60f8309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:46.201778 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1145/2279] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Aug 8 16:55:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:55:46.217961 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90edae0e-b483-40e1-8a2f-8caeafba169c None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:55:46.220361 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-90edae0e-b483-40e1-8a2f-8caeafba169c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:55:46.358912 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90edae0e-b483-40e1-8a2f-8caeafba169c None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:55:46.366556 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41774c9-cbee-4483-a58f-19ab726eddb1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c606decb832444e38a194b98f8d15f8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:55:46.369451 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41774c9-cbee-4483-a58f-19ab726eddb1 service swift] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:55:46.369451 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41774c9-cbee-4483-a58f-19ab726eddb1 service swift] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:55:46.369451 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41774c9-cbee-4483-a58f-19ab726eddb1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:55:46.379338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41774c9-cbee-4483-a58f-19ab726eddb1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:55:46.381086 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41774c9-cbee-4483-a58f-19ab726eddb1 service swift] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:55:46.393760 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1135/2280] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Aug 8 16:55:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 16:55:56.194429 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=25 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:55:56.386924 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=91,keystone:DELETE=2 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:56:15.693073 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5841f11-381c-4fb6-81e7-b0b2a60f8309 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:56:15.695422 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f5841f11-381c-4fb6-81e7-b0b2a60f8309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:15.810036 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5841f11-381c-4fb6-81e7-b0b2a60f8309 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:56:15.817016 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a42f52-fd43-4620-ab89-e51d1ae83185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:56:15.818104 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a42f52-fd43-4620-ab89-e51d1ae83185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:15.818594 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a42f52-fd43-4620-ab89-e51d1ae83185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:15.819092 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a42f52-fd43-4620-ab89-e51d1ae83185 admin admin] PATH_INFO: `/v3/users/aeb73c02c74046b5b0b85f8cb588455a` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:15.821456 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a42f52-fd43-4620-ab89-e51d1ae83185 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeb73c02c74046b5b0b85f8cb588455a)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:56:15.826937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a42f52-fd43-4620-ab89-e51d1ae83185 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:56:16.016387 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-59a42f52-fd43-4620-ab89-e51d1ae83185 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeb73c02c74046b5b0b85f8cb588455a'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:56:16.084806 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-59a42f52-fd43-4620-ab89-e51d1ae83185 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeb73c02c74046b5b0b85f8cb588455a'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:56:16.085536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-59a42f52-fd43-4620-ab89-e51d1ae83185 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeb73c02c74046b5b0b85f8cb588455a'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:56:16.095176 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-59a42f52-fd43-4620-ab89-e51d1ae83185 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeb73c02c74046b5b0b85f8cb588455a'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:56:16.111299 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-59a42f52-fd43-4620-ab89-e51d1ae83185 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeb73c02c74046b5b0b85f8cb588455a'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:56:16.118273 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1146/2281] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:56:15 2022] DELETE /identity/v3/users/aeb73c02c74046b5b0b85f8cb588455a => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:56:16.123992 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17158344-8d46-4a8a-8139-f15e73f2ade1 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:16.124750 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17158344-8d46-4a8a-8139-f15e73f2ade1 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:16.124750 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17158344-8d46-4a8a-8139-f15e73f2ade1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:16.146268 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-17158344-8d46-4a8a-8139-f15e73f2ade1 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:56:16.147613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-17158344-8d46-4a8a-8139-f15e73f2ade1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:16.220423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1136/2282] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:56:16.232197 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a42f52-fd43-4620-ab89-e51d1ae83185 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:56:16.234624 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-59a42f52-fd43-4620-ab89-e51d1ae83185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:16.316344 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a42f52-fd43-4620-ab89-e51d1ae83185 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:56:16.321204 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c0e06-767e-402a-ad59-d1e733ee1f56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:56:16.322089 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c0e06-767e-402a-ad59-d1e733ee1f56 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:16.322548 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c0e06-767e-402a-ad59-d1e733ee1f56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:16.323015 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c0e06-767e-402a-ad59-d1e733ee1f56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:16.334241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c0e06-767e-402a-ad59-d1e733ee1f56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:56:16.335860 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c0e06-767e-402a-ad59-d1e733ee1f56 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:56:16.348263 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1147/2283] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:56:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:56:16.889545 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17158344-8d46-4a8a-8139-f15e73f2ade1 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:56:16.891347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-17158344-8d46-4a8a-8139-f15e73f2ade1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:16.908067 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17158344-8d46-4a8a-8139-f15e73f2ade1 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:56:16.913461 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cce39d-9d87-45a5-b08c-c401187651ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:56:16.914131 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cce39d-9d87-45a5-b08c-c401187651ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:16.914411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cce39d-9d87-45a5-b08c-c401187651ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:16.914750 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cce39d-9d87-45a5-b08c-c401187651ce admin admin] PATH_INFO: `/v3/projects/1496f38f920d4bb995363e08ecbab3e7` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:16.915991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cce39d-9d87-45a5-b08c-c401187651ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=1496f38f920d4bb995363e08ecbab3e7)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:56:16.918992 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cce39d-9d87-45a5-b08c-c401187651ce admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:56:17.027636 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-91cce39d-9d87-45a5-b08c-c401187651ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1496f38f920d4bb995363e08ecbab3e7', 'request_id': 'req-91cce39d-9d87-45a5-b08c-c401187651ce'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:56:17.037814 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-91cce39d-9d87-45a5-b08c-c401187651ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1496f38f920d4bb995363e08ecbab3e7', 'request_id': 'req-91cce39d-9d87-45a5-b08c-c401187651ce'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:56:17.046097 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-91cce39d-9d87-45a5-b08c-c401187651ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1496f38f920d4bb995363e08ecbab3e7', 'request_id': 'req-91cce39d-9d87-45a5-b08c-c401187651ce'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:56:17.130351 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-91cce39d-9d87-45a5-b08c-c401187651ce admin admin] The token cache is being invalidate because project 1496f38f920d4bb995363e08ecbab3e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:56:17.131472 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-91cce39d-9d87-45a5-b08c-c401187651ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:56:17.131472 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-91cce39d-9d87-45a5-b08c-c401187651ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:56:17.132847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1137/2284] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:56:16 2022] DELETE /identity/v3/projects/1496f38f920d4bb995363e08ecbab3e7 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:56:26.343534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:56:27.126689 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:56:41.998486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8439b-3879-47cb-92fb-e7a93e016c1c None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:41.999223 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8439b-3879-47cb-92fb-e7a93e016c1c None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:41.999223 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8439b-3879-47cb-92fb-e7a93e016c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:42.020727 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-76f8439b-3879-47cb-92fb-e7a93e016c1c None None] MFA Rules not processed for user `683bed0160e54791a8e638523ac7e460`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:56:42.021993 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-76f8439b-3879-47cb-92fb-e7a93e016c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:42.087961 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1148/2285] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:56:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:56:46.487247 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cce39d-9d87-45a5-b08c-c401187651ce admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:56:46.489208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-91cce39d-9d87-45a5-b08c-c401187651ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:46.559506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cce39d-9d87-45a5-b08c-c401187651ce admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:56:46.564609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7986e1e-cac2-41b5-afed-334d2cfdd977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:56:46.565355 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7986e1e-cac2-41b5-afed-334d2cfdd977 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:46.565753 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7986e1e-cac2-41b5-afed-334d2cfdd977 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:46.566266 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7986e1e-cac2-41b5-afed-334d2cfdd977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:46.568850 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-c7986e1e-cac2-41b5-afed-334d2cfdd977 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:46.661266 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7986e1e-cac2-41b5-afed-334d2cfdd977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:56:46.664075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7986e1e-cac2-41b5-afed-334d2cfdd977 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:56:46.677449 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1138/2286] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:56:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:56:47.080187 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8439b-3879-47cb-92fb-e7a93e016c1c None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:56:47.082150 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-76f8439b-3879-47cb-92fb-e7a93e016c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:47.096768 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8439b-3879-47cb-92fb-e7a93e016c1c None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:56:47.101045 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1287ad-cbd2-482f-a1e0-1a26e1b6d53f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:56:47.101849 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1287ad-cbd2-482f-a1e0-1a26e1b6d53f service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:47.102237 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1287ad-cbd2-482f-a1e0-1a26e1b6d53f service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:47.102763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1287ad-cbd2-482f-a1e0-1a26e1b6d53f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:47.105915 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-0d1287ad-cbd2-482f-a1e0-1a26e1b6d53f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:47.124402 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1287ad-cbd2-482f-a1e0-1a26e1b6d53f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:56:47.127652 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1287ad-cbd2-482f-a1e0-1a26e1b6d53f service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:56:47.153111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1149/2287] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:56:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:56:54.036045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7986e1e-cac2-41b5-afed-334d2cfdd977 service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:56:54.048363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7986e1e-cac2-41b5-afed-334d2cfdd977 service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:56:54.052924 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca5e7e-804d-44bc-bdee-095420532e02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:56:54.053945 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca5e7e-804d-44bc-bdee-095420532e02 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:54.054386 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca5e7e-804d-44bc-bdee-095420532e02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:54.054938 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca5e7e-804d-44bc-bdee-095420532e02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:54.058654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-75ca5e7e-804d-44bc-bdee-095420532e02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:54.158626 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca5e7e-804d-44bc-bdee-095420532e02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:56:54.160101 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca5e7e-804d-44bc-bdee-095420532e02 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:56:54.172124 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1139/2288] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:56:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:56:57.141885 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=31 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:57:00.820642 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ddf0c-0b82-43ae-8f9f-ff204db7ce9f None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:00.822862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ddf0c-0b82-43ae-8f9f-ff204db7ce9f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:00.822862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ddf0c-0b82-43ae-8f9f-ff204db7ce9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:00.852535 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-bd5ddf0c-0b82-43ae-8f9f-ff204db7ce9f None None] MFA Rules not processed for user `9646662b0b3048f79ad5d2b9602f10a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:57:00.854911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-bd5ddf0c-0b82-43ae-8f9f-ff204db7ce9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:00.928305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1150/2289] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Aug 8 16:57:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:57:01.004946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca5e7e-804d-44bc-bdee-095420532e02 service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:57:01.007263 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-75ca5e7e-804d-44bc-bdee-095420532e02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:01.071964 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:57:01.110429 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca5e7e-804d-44bc-bdee-095420532e02 service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:57:01.115893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43fd98-d034-43b1-b4fd-5f596bce0a53 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c606decb832444e38a194b98f8d15f8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:57:01.119498 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43fd98-d034-43b1-b4fd-5f596bce0a53 service swift] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:01.119498 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43fd98-d034-43b1-b4fd-5f596bce0a53 service swift] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:01.119498 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43fd98-d034-43b1-b4fd-5f596bce0a53 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:01.128044 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43fd98-d034-43b1-b4fd-5f596bce0a53 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:57:01.129841 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43fd98-d034-43b1-b4fd-5f596bce0a53 service swift] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:57:01.140289 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1140/2290] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Aug 8 16:57:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 16:57:03.460733 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ddf0c-0b82-43ae-8f9f-ff204db7ce9f None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:57:03.463412 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-bd5ddf0c-0b82-43ae-8f9f-ff204db7ce9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:03.482853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ddf0c-0b82-43ae-8f9f-ff204db7ce9f None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:57:03.488860 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14b6f7-6aaf-4493-8596-200993503566 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:57:03.489848 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14b6f7-6aaf-4493-8596-200993503566 service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:03.490130 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14b6f7-6aaf-4493-8596-200993503566 service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:03.490404 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14b6f7-6aaf-4493-8596-200993503566 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:03.492721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-9d14b6f7-6aaf-4493-8596-200993503566 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:03.514277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43fd98-d034-43b1-b4fd-5f596bce0a53 service swift] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:57:03.517356 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ea43fd98-d034-43b1-b4fd-5f596bce0a53 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:03.605671 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d14b6f7-6aaf-4493-8596-200993503566 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:57:03.608020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d14b6f7-6aaf-4493-8596-200993503566 service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:57:03.619088 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1151/2291] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:57:03 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:57:03.716368 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43fd98-d034-43b1-b4fd-5f596bce0a53 service swift] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:57:03.722239 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a2a33-489d-4860-8c06-bd9c285aaae7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:57:03.723245 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a2a33-489d-4860-8c06-bd9c285aaae7 service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:03.723673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a2a33-489d-4860-8c06-bd9c285aaae7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:03.724102 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a2a33-489d-4860-8c06-bd9c285aaae7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:03.727390 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-8b0a2a33-489d-4860-8c06-bd9c285aaae7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:03.900537 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a2a33-489d-4860-8c06-bd9c285aaae7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:57:03.903576 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a2a33-489d-4860-8c06-bd9c285aaae7 service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:57:03.918250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1141/2292] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:57:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:57:04.187845 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14b6f7-6aaf-4493-8596-200993503566 service nova] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:57:04.190026 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-9d14b6f7-6aaf-4493-8596-200993503566 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:04.222464 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14b6f7-6aaf-4493-8596-200993503566 service nova] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:57:04.231369 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f55f58d-c5fc-4bed-aa36-cc69aa87c884 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:57:04.232089 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55f58d-c5fc-4bed-aa36-cc69aa87c884 service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:04.232579 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55f58d-c5fc-4bed-aa36-cc69aa87c884 service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:04.232952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55f58d-c5fc-4bed-aa36-cc69aa87c884 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:04.235861 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-8f55f58d-c5fc-4bed-aa36-cc69aa87c884 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:04.379117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f55f58d-c5fc-4bed-aa36-cc69aa87c884 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:57:04.382207 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f55f58d-c5fc-4bed-aa36-cc69aa87c884 service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:57:04.402645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1152/2293] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:57:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:57:13.909274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=64 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:57:14.392667 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=81 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:57:21.758640 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a2a33-489d-4860-8c06-bd9c285aaae7 service cinder] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:57:21.770483 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a2a33-489d-4860-8c06-bd9c285aaae7 service cinder] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:57:21.774516 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50064fed-7bb2-428c-ad32-9e55315c38a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:57:21.775175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50064fed-7bb2-428c-ad32-9e55315c38a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:21.775450 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50064fed-7bb2-428c-ad32-9e55315c38a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:21.775720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50064fed-7bb2-428c-ad32-9e55315c38a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:21.777621 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-50064fed-7bb2-428c-ad32-9e55315c38a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:21.789170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50064fed-7bb2-428c-ad32-9e55315c38a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:57:21.793746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50064fed-7bb2-428c-ad32-9e55315c38a8 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:57:21.800223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1142/2294] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:57:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:57:29.594663 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f55f58d-c5fc-4bed-aa36-cc69aa87c884 service nova] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:57:29.606327 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f55f58d-c5fc-4bed-aa36-cc69aa87c884 service nova] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:57:29.610265 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3607cab-6596-43be-8bc0-971652aba541 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:57:29.610954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3607cab-6596-43be-8bc0-971652aba541 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:29.611206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3607cab-6596-43be-8bc0-971652aba541 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:29.611531 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3607cab-6596-43be-8bc0-971652aba541 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:29.614011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-e3607cab-6596-43be-8bc0-971652aba541 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:29.627481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3607cab-6596-43be-8bc0-971652aba541 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:57:29.629273 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3607cab-6596-43be-8bc0-971652aba541 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:57:29.640595 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1153/2295] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:57:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:57:30.193373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645878b8-a9ff-4eb5-9ced-01c0f6853b6f None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:30.193373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645878b8-a9ff-4eb5-9ced-01c0f6853b6f None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:30.193373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645878b8-a9ff-4eb5-9ced-01c0f6853b6f None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:30.193373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1143/2296] 10.222.0.13 () {54 vars in 921 bytes} [Mon Aug 8 16:57:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:57:30.197012 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b53a3a-cac3-474b-92db-bf08e929b8dd None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:30.197478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b53a3a-cac3-474b-92db-bf08e929b8dd None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:30.197936 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b53a3a-cac3-474b-92db-bf08e929b8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:30.240084 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-b8b53a3a-cac3-474b-92db-bf08e929b8dd None None] MFA Rules not processed for user `9daa7489bced467984d39e8b8ac49d13`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:57:30.242339 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b8b53a3a-cac3-474b-92db-bf08e929b8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:30.253548 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1154/2297] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Aug 8 16:57:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:57:30.268504 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645878b8-a9ff-4eb5-9ced-01c0f6853b6f None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:57:30.270663 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-645878b8-a9ff-4eb5-9ced-01c0f6853b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:30.437352 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645878b8-a9ff-4eb5-9ced-01c0f6853b6f None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:57:30.446386 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef40616-fdb1-4645-8a92-07da0770f96a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea51ef02128a4b63b11049eb0daef298', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:57:30.447477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef40616-fdb1-4645-8a92-07da0770f96a service glance] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:30.447980 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef40616-fdb1-4645-8a92-07da0770f96a service glance] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:30.448408 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef40616-fdb1-4645-8a92-07da0770f96a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:30.466717 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef40616-fdb1-4645-8a92-07da0770f96a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:57:30.468244 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef40616-fdb1-4645-8a92-07da0770f96a service glance] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:57:30.483003 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1144/2298] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:57:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:57:30.744468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7dd6c-f01a-43f7-91f1-ebcbdd6eb632 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:30.745461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7dd6c-f01a-43f7-91f1-ebcbdd6eb632 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:30.745461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7dd6c-f01a-43f7-91f1-ebcbdd6eb632 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:30.747039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1155/2299] 10.222.0.13 () {54 vars in 921 bytes} [Mon Aug 8 16:57:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:57:30.752866 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0349a66-b553-4ee0-97a1-e9e5b8b554bb None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:30.753129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0349a66-b553-4ee0-97a1-e9e5b8b554bb None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:30.753400 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0349a66-b553-4ee0-97a1-e9e5b8b554bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:30.778538 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-d0349a66-b553-4ee0-97a1-e9e5b8b554bb None None] MFA Rules not processed for user `9482bf89c1b648c1a91ee00ee5fbba63`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:57:30.779871 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d0349a66-b553-4ee0-97a1-e9e5b8b554bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:30.787148 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1145/2300] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Aug 8 16:57:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:57:30.800066 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7dd6c-f01a-43f7-91f1-ebcbdd6eb632 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:57:30.816111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7dd6c-f01a-43f7-91f1-ebcbdd6eb632 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:57:30.821157 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746ac4f-e411-45bd-965d-873f55b36c3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:57:30.821782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746ac4f-e411-45bd-965d-873f55b36c3a service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:30.830062 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746ac4f-e411-45bd-965d-873f55b36c3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:30.830062 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746ac4f-e411-45bd-965d-873f55b36c3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:30.832136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746ac4f-e411-45bd-965d-873f55b36c3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:57:30.833378 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746ac4f-e411-45bd-965d-873f55b36c3a service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:57:30.843919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1156/2301] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:57:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:57:30.973998 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd07cc-3f0c-43c7-8c69-a26488ddf0ef None None] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:30.974300 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd07cc-3f0c-43c7-8c69-a26488ddf0ef None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:30.974615 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd07cc-3f0c-43c7-8c69-a26488ddf0ef None None] PATH_INFO: `/` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:30.976182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1146/2302] 10.222.0.13 () {54 vars in 921 bytes} [Mon Aug 8 16:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:57:30.984927 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84473a18-0008-4ff1-94e8-3a162164eba2 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:30.985362 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84473a18-0008-4ff1-94e8-3a162164eba2 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:30.985725 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84473a18-0008-4ff1-94e8-3a162164eba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:31.011822 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-84473a18-0008-4ff1-94e8-3a162164eba2 None None] MFA Rules not processed for user `9daa7489bced467984d39e8b8ac49d13`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:57:31.013357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-84473a18-0008-4ff1-94e8-3a162164eba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:31.022987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1157/2303] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Aug 8 16:57:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:57:31.035166 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd07cc-3f0c-43c7-8c69-a26488ddf0ef None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:57:31.047536 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd07cc-3f0c-43c7-8c69-a26488ddf0ef None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:57:31.051193 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642db05-a069-4f61-bd6b-abffcbbfe47d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:57:31.051891 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642db05-a069-4f61-bd6b-abffcbbfe47d service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:31.052072 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642db05-a069-4f61-bd6b-abffcbbfe47d service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:31.052339 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642db05-a069-4f61-bd6b-abffcbbfe47d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:31.064816 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642db05-a069-4f61-bd6b-abffcbbfe47d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:57:31.066454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642db05-a069-4f61-bd6b-abffcbbfe47d service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:57:31.080512 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1147/2304] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:57:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:57:31.153264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84473a18-0008-4ff1-94e8-3a162164eba2 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:57:31.154709 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-84473a18-0008-4ff1-94e8-3a162164eba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:31.236416 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84473a18-0008-4ff1-94e8-3a162164eba2 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:57:31.245796 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a553d-47a8-4f3f-8f1b-5f72b2700533 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e74dbb121d749749cf5f9d9d7221008', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:57:31.247039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a553d-47a8-4f3f-8f1b-5f72b2700533 service placement] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:31.247482 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a553d-47a8-4f3f-8f1b-5f72b2700533 service placement] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:31.247968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a553d-47a8-4f3f-8f1b-5f72b2700533 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:31.251182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b03a553d-47a8-4f3f-8f1b-5f72b2700533 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:31.268272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a553d-47a8-4f3f-8f1b-5f72b2700533 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:57:31.270153 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a553d-47a8-4f3f-8f1b-5f72b2700533 service placement] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:57:31.282694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1158/2305] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:57:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:57:36.397333 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642db05-a069-4f61-bd6b-abffcbbfe47d service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:57:36.413208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642db05-a069-4f61-bd6b-abffcbbfe47d service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:57:36.419543 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9cdef-ce95-4432-8525-d9c31f63be00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:57:36.420174 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9cdef-ce95-4432-8525-d9c31f63be00 service nova] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:36.420450 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9cdef-ce95-4432-8525-d9c31f63be00 service nova] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:36.420721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9cdef-ce95-4432-8525-d9c31f63be00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:36.422857 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d8e9cdef-ce95-4432-8525-d9c31f63be00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:36.439486 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9cdef-ce95-4432-8525-d9c31f63be00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:57:36.442549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9cdef-ce95-4432-8525-d9c31f63be00 service nova] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:57:36.469967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1148/2306] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:57:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:57:37.768143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a553d-47a8-4f3f-8f1b-5f72b2700533 service placement] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:57:37.806637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a553d-47a8-4f3f-8f1b-5f72b2700533 service placement] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:57:37.816146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc04ce-9201-467b-b6d7-5e7fd89acb63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:57:37.817834 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc04ce-9201-467b-b6d7-5e7fd89acb63 service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:37.818417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc04ce-9201-467b-b6d7-5e7fd89acb63 service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:37.819041 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc04ce-9201-467b-b6d7-5e7fd89acb63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:37.824362 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-0bcc04ce-9201-467b-b6d7-5e7fd89acb63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:37.857727 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc04ce-9201-467b-b6d7-5e7fd89acb63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:57:37.863200 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc04ce-9201-467b-b6d7-5e7fd89acb63 service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:57:37.904921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1159/2307] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:57:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:57:40.829234 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc104a81-203e-4282-a741-491faf196e11 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:40.830158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc104a81-203e-4282-a741-491faf196e11 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:40.830158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc104a81-203e-4282-a741-491faf196e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:40.874879 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-fc104a81-203e-4282-a741-491faf196e11 None None] MFA Rules not processed for user `9646662b0b3048f79ad5d2b9602f10a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:57:40.877256 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-fc104a81-203e-4282-a741-491faf196e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:40.891295 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1149/2308] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Aug 8 16:57:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:57:40.913012 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc04ce-9201-467b-b6d7-5e7fd89acb63 service nova] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:57:40.935185 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc04ce-9201-467b-b6d7-5e7fd89acb63 service nova] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:57:40.953346 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f20fce-1a8a-4579-b87a-b37561d0143f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c606decb832444e38a194b98f8d15f8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:57:40.961040 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f20fce-1a8a-4579-b87a-b37561d0143f service swift] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:40.961040 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f20fce-1a8a-4579-b87a-b37561d0143f service swift] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:40.961040 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f20fce-1a8a-4579-b87a-b37561d0143f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:40.971198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f20fce-1a8a-4579-b87a-b37561d0143f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:57:40.973104 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f20fce-1a8a-4579-b87a-b37561d0143f service swift] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:57:40.989126 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1160/2309] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Aug 8 16:57:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 16:57:46.030902 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3abbd-ac32-4c38-bce3-2af5fd86b27a None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:46.031565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3abbd-ac32-4c38-bce3-2af5fd86b27a None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:46.031565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3abbd-ac32-4c38-bce3-2af5fd86b27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:46.045318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=51 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:57:46.061430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-09e3abbd-ac32-4c38-bce3-2af5fd86b27a None None] MFA Rules not processed for user `fdb16e7edbe143e9b96325842896f04e`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:57:46.062781 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-09e3abbd-ac32-4c38-bce3-2af5fd86b27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:46.071796 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1150/2310] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:57:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:57:46.635645 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f20fce-1a8a-4579-b87a-b37561d0143f service swift] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:57:46.657382 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f20fce-1a8a-4579-b87a-b37561d0143f service swift] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:57:46.664549 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b55b30-fede-4777-a441-ec42572bb416 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:57:46.664549 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b55b30-fede-4777-a441-ec42572bb416 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:46.666417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b55b30-fede-4777-a441-ec42572bb416 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:46.666972 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b55b30-fede-4777-a441-ec42572bb416 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:46.669713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-e9b55b30-fede-4777-a441-ec42572bb416 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:46.693742 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b55b30-fede-4777-a441-ec42572bb416 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:57:46.704373 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b55b30-fede-4777-a441-ec42572bb416 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:57:46.712240 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1161/2311] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:57:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:57:56.081833 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=2 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:57:56.702884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=68 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:58:01.474135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732524b-3c36-4743-a53a-cafd5966df64 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:01.475117 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732524b-3c36-4743-a53a-cafd5966df64 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:01.475306 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732524b-3c36-4743-a53a-cafd5966df64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:01.503528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-1732524b-3c36-4743-a53a-cafd5966df64 None None] MFA Rules not processed for user `75d793a42ec74232899b7a19e36b5867`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:58:01.504995 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-1732524b-3c36-4743-a53a-cafd5966df64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:01.514622 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1151/2312] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:58:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:58:07.668540 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b55b30-fede-4777-a441-ec42572bb416 service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:07.684316 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b55b30-fede-4777-a441-ec42572bb416 service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:07.690165 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38551b8f-c193-471f-8db7-d3891875d9fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:07.691363 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38551b8f-c193-471f-8db7-d3891875d9fa service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:07.691527 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38551b8f-c193-471f-8db7-d3891875d9fa service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:07.691811 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38551b8f-c193-471f-8db7-d3891875d9fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:07.694497 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-38551b8f-c193-471f-8db7-d3891875d9fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:07.708141 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38551b8f-c193-471f-8db7-d3891875d9fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:07.710715 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38551b8f-c193-471f-8db7-d3891875d9fa service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:07.728154 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1162/2313] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:58:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:58:08.970119 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732524b-3c36-4743-a53a-cafd5966df64 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:08.971967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-1732524b-3c36-4743-a53a-cafd5966df64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:09.078997 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732524b-3c36-4743-a53a-cafd5966df64 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:09.090136 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b18f0cc-a005-4379-ade1-becfd892b425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:09.091256 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b18f0cc-a005-4379-ade1-becfd892b425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:09.091760 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b18f0cc-a005-4379-ade1-becfd892b425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:09.092253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b18f0cc-a005-4379-ade1-becfd892b425 admin admin] PATH_INFO: `/v3/users/683bed0160e54791a8e638523ac7e460` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:09.094373 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b18f0cc-a005-4379-ade1-becfd892b425 admin admin] RBAC: Authorizing `identity:delete_user(user_id=683bed0160e54791a8e638523ac7e460)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:09.098800 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b18f0cc-a005-4379-ade1-becfd892b425 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:09.371805 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7b18f0cc-a005-4379-ade1-becfd892b425 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '683bed0160e54791a8e638523ac7e460'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:58:09.601779 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7b18f0cc-a005-4379-ade1-becfd892b425 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '683bed0160e54791a8e638523ac7e460'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:58:09.603102 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7b18f0cc-a005-4379-ade1-becfd892b425 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '683bed0160e54791a8e638523ac7e460'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:58:09.622322 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7b18f0cc-a005-4379-ade1-becfd892b425 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '683bed0160e54791a8e638523ac7e460'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:58:09.626871 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7b18f0cc-a005-4379-ade1-becfd892b425 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '683bed0160e54791a8e638523ac7e460'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:58:09.663848 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1152/2314] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:58:08 2022] DELETE /identity/v3/users/683bed0160e54791a8e638523ac7e460 => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:58:10.111800 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38551b8f-c193-471f-8db7-d3891875d9fa service nova] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:10.129653 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38551b8f-c193-471f-8db7-d3891875d9fa service nova] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:10.220643 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84aedcb-88a9-407e-ad1b-777e281c1166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:10.221761 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84aedcb-88a9-407e-ad1b-777e281c1166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:10.222240 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84aedcb-88a9-407e-ad1b-777e281c1166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:10.222778 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84aedcb-88a9-407e-ad1b-777e281c1166 admin admin] PATH_INFO: `/v3/projects/49844b73a63b4c338fc5109f70ba774e` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:10.224949 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84aedcb-88a9-407e-ad1b-777e281c1166 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49844b73a63b4c338fc5109f70ba774e)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:10.228814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84aedcb-88a9-407e-ad1b-777e281c1166 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:10.405879 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-d84aedcb-88a9-407e-ad1b-777e281c1166 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49844b73a63b4c338fc5109f70ba774e', 'request_id': 'req-d84aedcb-88a9-407e-ad1b-777e281c1166'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:58:10.504407 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-d84aedcb-88a9-407e-ad1b-777e281c1166 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49844b73a63b4c338fc5109f70ba774e', 'request_id': 'req-d84aedcb-88a9-407e-ad1b-777e281c1166'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:58:10.516679 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-d84aedcb-88a9-407e-ad1b-777e281c1166 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49844b73a63b4c338fc5109f70ba774e', 'request_id': 'req-d84aedcb-88a9-407e-ad1b-777e281c1166'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:58:10.539932 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-d84aedcb-88a9-407e-ad1b-777e281c1166 admin admin] The token cache is being invalidate because project 49844b73a63b4c338fc5109f70ba774e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:58:10.540258 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-d84aedcb-88a9-407e-ad1b-777e281c1166 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:58:10.541117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-d84aedcb-88a9-407e-ad1b-777e281c1166 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:58:10.543570 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1163/2315] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:58:10 2022] DELETE /identity/v3/projects/49844b73a63b4c338fc5109f70ba774e => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:58:10.550527 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c619c-fdc9-4535-bd40-f9a759e0ef9a None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:10.551140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c619c-fdc9-4535-bd40-f9a759e0ef9a None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:10.551140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c619c-fdc9-4535-bd40-f9a759e0ef9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:10.575157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-665c619c-fdc9-4535-bd40-f9a759e0ef9a None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:58:10.576868 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-665c619c-fdc9-4535-bd40-f9a759e0ef9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:10.687589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1153/2316] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:58:10.757339 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f26da2-6df9-4e99-b316-197f62a8a3bd None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:10.757634 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f26da2-6df9-4e99-b316-197f62a8a3bd None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:10.757907 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f26da2-6df9-4e99-b316-197f62a8a3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:10.780692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-e9f26da2-6df9-4e99-b316-197f62a8a3bd None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:58:10.781908 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-e9f26da2-6df9-4e99-b316-197f62a8a3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:10.800347 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1164/2317] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:58:10.802076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c619c-fdc9-4535-bd40-f9a759e0ef9a None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:10.814627 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c619c-fdc9-4535-bd40-f9a759e0ef9a None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:10.821472 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64eca45-ce10-4258-b3fc-491d9d606072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:10.822409 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64eca45-ce10-4258-b3fc-491d9d606072 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:10.822733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64eca45-ce10-4258-b3fc-491d9d606072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:10.823003 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64eca45-ce10-4258-b3fc-491d9d606072 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:10.823729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64eca45-ce10-4258-b3fc-491d9d606072 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:58:10.824066 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64eca45-ce10-4258-b3fc-491d9d606072 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:10.825815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64eca45-ce10-4258-b3fc-491d9d606072 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:10.839693 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:58:10.850678 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1154/2318] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:58:10.852271 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f26da2-6df9-4e99-b316-197f62a8a3bd None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:10.873575 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f26da2-6df9-4e99-b316-197f62a8a3bd None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:10.882804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b616f3b-540b-49c0-86c8-ba3331fc7599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:10.883887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b616f3b-540b-49c0-86c8-ba3331fc7599 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:10.884379 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b616f3b-540b-49c0-86c8-ba3331fc7599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:10.884855 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b616f3b-540b-49c0-86c8-ba3331fc7599 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:10.886244 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b616f3b-540b-49c0-86c8-ba3331fc7599 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:10.890279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b616f3b-540b-49c0-86c8-ba3331fc7599 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:11.017823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1165/2319] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:58:10 2022] POST /identity/v3/projects => generated 371 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:58:11.020534 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64eca45-ce10-4258-b3fc-491d9d606072 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:11.041697 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64eca45-ce10-4258-b3fc-491d9d606072 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:11.090456 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7128c-936e-4509-8bbe-b52af39c9769 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:11.090962 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7128c-936e-4509-8bbe-b52af39c9769 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:11.091429 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7128c-936e-4509-8bbe-b52af39c9769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:11.143315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-75c7128c-936e-4509-8bbe-b52af39c9769 None None] MFA Rules not processed for user `9646662b0b3048f79ad5d2b9602f10a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:58:11.146700 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-75c7128c-936e-4509-8bbe-b52af39c9769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:11.262360 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeac4b2-3fcb-41b0-b7b7-91a014b50063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:11.265667 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeac4b2-3fcb-41b0-b7b7-91a014b50063 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:11.267458 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeac4b2-3fcb-41b0-b7b7-91a014b50063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:11.268941 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeac4b2-3fcb-41b0-b7b7-91a014b50063 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:11.271418 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeac4b2-3fcb-41b0-b7b7-91a014b50063 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:11.277524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeac4b2-3fcb-41b0-b7b7-91a014b50063 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:11.277862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:58:11.309329 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1166/2320] 10.222.0.13 () {58 vars in 967 bytes} [Mon Aug 8 16:58:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:58:11.377989 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7128c-936e-4509-8bbe-b52af39c9769 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:11.381375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-75c7128c-936e-4509-8bbe-b52af39c9769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:11.455211 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1155/2321] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:58:11 2022] POST /identity/v3/users => generated 286 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:58:11.468374 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeac4b2-3fcb-41b0-b7b7-91a014b50063 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:11.494681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeac4b2-3fcb-41b0-b7b7-91a014b50063 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:11.511815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6ec2e-10a3-4228-a59c-8a04a8b81aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:11.513853 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6ec2e-10a3-4228-a59c-8a04a8b81aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:11.518172 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6ec2e-10a3-4228-a59c-8a04a8b81aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:11.526991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6ec2e-10a3-4228-a59c-8a04a8b81aa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:11.528045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6ec2e-10a3-4228-a59c-8a04a8b81aa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:11.530895 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6ec2e-10a3-4228-a59c-8a04a8b81aa0 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:11.550842 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1156/2322] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:58:11.556112 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6ec2e-10a3-4228-a59c-8a04a8b81aa0 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:11.574524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6ec2e-10a3-4228-a59c-8a04a8b81aa0 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:11.581397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63073cc-7fef-4315-a98f-fff133226765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:11.587679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63073cc-7fef-4315-a98f-fff133226765 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:11.589832 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63073cc-7fef-4315-a98f-fff133226765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:11.590977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63073cc-7fef-4315-a98f-fff133226765 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:11.592376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63073cc-7fef-4315-a98f-fff133226765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:11.596600 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63073cc-7fef-4315-a98f-fff133226765 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:11.605681 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7128c-936e-4509-8bbe-b52af39c9769 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:11.615299 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d65d8a-c5fc-4f16-8a32-05abad2691ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c606decb832444e38a194b98f8d15f8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:11.616956 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d65d8a-c5fc-4f16-8a32-05abad2691ed service swift] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:11.617450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d65d8a-c5fc-4f16-8a32-05abad2691ed service swift] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:11.618124 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d65d8a-c5fc-4f16-8a32-05abad2691ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:11.648306 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d65d8a-c5fc-4f16-8a32-05abad2691ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:11.652870 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d65d8a-c5fc-4f16-8a32-05abad2691ed service swift] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:11.654193 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1157/2323] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:58:11.659043 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63073cc-7fef-4315-a98f-fff133226765 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:11.674183 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1167/2324] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Aug 8 16:58:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 16:58:11.681216 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63073cc-7fef-4315-a98f-fff133226765 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:11.687669 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155a4ff-b5df-4d7e-a642-abc644fd9dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:11.689378 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155a4ff-b5df-4d7e-a642-abc644fd9dee admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:11.689877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155a4ff-b5df-4d7e-a642-abc644fd9dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:11.691225 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155a4ff-b5df-4d7e-a642-abc644fd9dee admin admin] PATH_INFO: `/v3/projects/73ca0765d1fb44cda1c4c2562a695ac6/users/803b15aebfb044c3893831f84f1aa1d7/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:11.715926 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155a4ff-b5df-4d7e-a642-abc644fd9dee admin admin] RBAC: Authorizing `identity:create_grant(project_id=73ca0765d1fb44cda1c4c2562a695ac6, user_id=803b15aebfb044c3893831f84f1aa1d7, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:11.722211 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155a4ff-b5df-4d7e-a642-abc644fd9dee admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:11.828504 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1158/2325] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:58:11 2022] PUT /identity/v3/projects/73ca0765d1fb44cda1c4c2562a695ac6/users/803b15aebfb044c3893831f84f1aa1d7/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:58:11.835745 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c62820-d8f3-4b1d-886c-cdae35eae6c6 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:11.836120 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c62820-d8f3-4b1d-886c-cdae35eae6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:11.836709 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c62820-d8f3-4b1d-886c-cdae35eae6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:11.873808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-32c62820-d8f3-4b1d-886c-cdae35eae6c6 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:58:11.876388 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-32c62820-d8f3-4b1d-886c-cdae35eae6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:11.989477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1168/2326] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:58:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:58:12.008754 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155a4ff-b5df-4d7e-a642-abc644fd9dee admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:12.009707 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-6155a4ff-b5df-4d7e-a642-abc644fd9dee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:12.096806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155a4ff-b5df-4d7e-a642-abc644fd9dee admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:12.101995 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e8072-97ff-45de-85d8-7380d6e97370 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:12.102689 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e8072-97ff-45de-85d8-7380d6e97370 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:12.103077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e8072-97ff-45de-85d8-7380d6e97370 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:12.103406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e8072-97ff-45de-85d8-7380d6e97370 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:12.114475 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e8072-97ff-45de-85d8-7380d6e97370 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:12.116736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e8072-97ff-45de-85d8-7380d6e97370 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:12.129780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1159/2327] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:58:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:58:21.196257 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9e8bd-3921-49be-8aed-8e6b16170c7d None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:21.196855 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9e8bd-3921-49be-8aed-8e6b16170c7d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:21.197053 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9e8bd-3921-49be-8aed-8e6b16170c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:21.245037 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-dbc9e8bd-3921-49be-8aed-8e6b16170c7d None None] MFA Rules not processed for user `803b15aebfb044c3893831f84f1aa1d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:58:21.248949 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-dbc9e8bd-3921-49be-8aed-8e6b16170c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:21.373875 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1169/2328] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:58:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:58:21.374549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e8072-97ff-45de-85d8-7380d6e97370 service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:21.392154 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e8072-97ff-45de-85d8-7380d6e97370 service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:21.400671 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f1c3e-c8b7-43d7-b819-3b823c0c6876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:21.401947 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f1c3e-c8b7-43d7-b819-3b823c0c6876 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:21.402421 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f1c3e-c8b7-43d7-b819-3b823c0c6876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:21.402957 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f1c3e-c8b7-43d7-b819-3b823c0c6876 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:21.404063 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f1c3e-c8b7-43d7-b819-3b823c0c6876 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:21.407827 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f1c3e-c8b7-43d7-b819-3b823c0c6876 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:21.568664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1160/2329] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:58:21 2022] POST /identity/v3/projects => generated 373 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:58:21.570026 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9e8bd-3921-49be-8aed-8e6b16170c7d None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:21.586299 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9e8bd-3921-49be-8aed-8e6b16170c7d None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:21.597434 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:58:21.669162 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff33370-b6cd-437a-8a3a-4f8dab993f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:21.670379 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff33370-b6cd-437a-8a3a-4f8dab993f2c admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:21.671011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff33370-b6cd-437a-8a3a-4f8dab993f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:21.671499 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff33370-b6cd-437a-8a3a-4f8dab993f2c admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:21.672660 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff33370-b6cd-437a-8a3a-4f8dab993f2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:21.676708 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff33370-b6cd-437a-8a3a-4f8dab993f2c admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:21.782772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1170/2330] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:58:21 2022] POST /identity/v3/users => generated 293 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:58:21.789204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f1c3e-c8b7-43d7-b819-3b823c0c6876 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:21.800948 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f1c3e-c8b7-43d7-b819-3b823c0c6876 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:21.807012 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c25e4-f316-489c-804a-c6f27cc14ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:21.807639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c25e4-f316-489c-804a-c6f27cc14ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:21.807913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c25e4-f316-489c-804a-c6f27cc14ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:21.808178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c25e4-f316-489c-804a-c6f27cc14ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:21.808829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c25e4-f316-489c-804a-c6f27cc14ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:21.810468 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c25e4-f316-489c-804a-c6f27cc14ae3 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:21.823892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1161/2331] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:58:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:58:21.828583 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff33370-b6cd-437a-8a3a-4f8dab993f2c admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:21.840694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff33370-b6cd-437a-8a3a-4f8dab993f2c admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:21.846113 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c1b13-b81a-49f0-841f-71612a34e002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:21.846901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c1b13-b81a-49f0-841f-71612a34e002 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:21.847175 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c1b13-b81a-49f0-841f-71612a34e002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:21.847453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c1b13-b81a-49f0-841f-71612a34e002 admin admin] PATH_INFO: `/v3/projects/b8b7a3ba3c29499eb4459d4db10bf0f8/users/d24fd81503954a6e94d2eec0062882ed/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:21.867000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c1b13-b81a-49f0-841f-71612a34e002 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8b7a3ba3c29499eb4459d4db10bf0f8, user_id=d24fd81503954a6e94d2eec0062882ed, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:21.869711 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c1b13-b81a-49f0-841f-71612a34e002 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:21.952714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1171/2332] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:58:21 2022] PUT /identity/v3/projects/b8b7a3ba3c29499eb4459d4db10bf0f8/users/d24fd81503954a6e94d2eec0062882ed/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:58:21.957327 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c25e4-f316-489c-804a-c6f27cc14ae3 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:21.972334 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c25e4-f316-489c-804a-c6f27cc14ae3 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:21.984217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:58:22.051066 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad04d16-d873-4da7-a072-c244e19fea3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:22.051734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad04d16-d873-4da7-a072-c244e19fea3e admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:22.052017 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad04d16-d873-4da7-a072-c244e19fea3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:22.052456 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad04d16-d873-4da7-a072-c244e19fea3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:22.054098 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad04d16-d873-4da7-a072-c244e19fea3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:22.057487 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad04d16-d873-4da7-a072-c244e19fea3e admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:22.079929 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1162/2333] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:58:21 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:58:22.082037 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c1b13-b81a-49f0-841f-71612a34e002 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:22.104425 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c1b13-b81a-49f0-841f-71612a34e002 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:22.112992 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a629f5f-edb2-4a20-9892-bca9d91d2dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:22.115949 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a629f5f-edb2-4a20-9892-bca9d91d2dce admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:22.115949 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a629f5f-edb2-4a20-9892-bca9d91d2dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:22.116179 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a629f5f-edb2-4a20-9892-bca9d91d2dce admin admin] PATH_INFO: `/v3/projects/b8b7a3ba3c29499eb4459d4db10bf0f8/users/d24fd81503954a6e94d2eec0062882ed/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:22.119108 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a629f5f-edb2-4a20-9892-bca9d91d2dce admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8b7a3ba3c29499eb4459d4db10bf0f8, user_id=d24fd81503954a6e94d2eec0062882ed, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:22.121948 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a629f5f-edb2-4a20-9892-bca9d91d2dce admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:22.134868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1172/2334] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:58:22 2022] PUT /identity/v3/projects/b8b7a3ba3c29499eb4459d4db10bf0f8/users/d24fd81503954a6e94d2eec0062882ed/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:58:28.944872 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763dcb00-aeb1-44d4-b5a6-c15932a176b6 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:28.945682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763dcb00-aeb1-44d4-b5a6-c15932a176b6 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:28.945682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763dcb00-aeb1-44d4-b5a6-c15932a176b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:28.996798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-763dcb00-aeb1-44d4-b5a6-c15932a176b6 None None] MFA Rules not processed for user `d24fd81503954a6e94d2eec0062882ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:58:28.998561 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-763dcb00-aeb1-44d4-b5a6-c15932a176b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:29.137957 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1163/2335] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:58:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:58:29.160835 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a629f5f-edb2-4a20-9892-bca9d91d2dce admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:29.164966 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-0a629f5f-edb2-4a20-9892-bca9d91d2dce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:29.294283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a629f5f-edb2-4a20-9892-bca9d91d2dce admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:29.302532 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ddebfa-a360-4b73-8772-07bda15052d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:29.303252 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddebfa-a360-4b73-8772-07bda15052d1 service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:29.303533 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddebfa-a360-4b73-8772-07bda15052d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:29.303872 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddebfa-a360-4b73-8772-07bda15052d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:29.382548 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ddebfa-a360-4b73-8772-07bda15052d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:29.384200 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ddebfa-a360-4b73-8772-07bda15052d1 service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:29.394612 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1173/2336] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:58:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:58:33.006790 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c871e-b0dd-4079-9d17-8e3da814831e None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:33.008167 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c871e-b0dd-4079-9d17-8e3da814831e None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:33.008652 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c871e-b0dd-4079-9d17-8e3da814831e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:33.062850 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-ea1c871e-b0dd-4079-9d17-8e3da814831e None None] MFA Rules not processed for user `803b15aebfb044c3893831f84f1aa1d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:58:33.065944 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ea1c871e-b0dd-4079-9d17-8e3da814831e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:33.090048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1164/2337] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:58:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:58:39.120850 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c45988-c7f4-45e2-ba7a-64eed9f7c9de None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:39.121448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c45988-c7f4-45e2-ba7a-64eed9f7c9de None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:39.121448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c45988-c7f4-45e2-ba7a-64eed9f7c9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:39.129046 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:58:39.144886 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-b1c45988-c7f4-45e2-ba7a-64eed9f7c9de None None] MFA Rules not processed for user `5a8db3ff64ca44958fd633ba25ee191f`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:58:39.146558 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b1c45988-c7f4-45e2-ba7a-64eed9f7c9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:39.224739 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1174/2338] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:58:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:58:43.038931 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=60 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:58:49.215722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=24 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:59:07.185719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c871e-b0dd-4079-9d17-8e3da814831e None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:07.210879 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c871e-b0dd-4079-9d17-8e3da814831e None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:07.220493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc93f94-9140-47a7-87e5-8e091010034b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:07.221542 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc93f94-9140-47a7-87e5-8e091010034b service nova] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:07.222025 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc93f94-9140-47a7-87e5-8e091010034b service nova] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:07.222538 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc93f94-9140-47a7-87e5-8e091010034b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:07.225683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-2cc93f94-9140-47a7-87e5-8e091010034b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:07.304201 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc93f94-9140-47a7-87e5-8e091010034b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:07.305838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc93f94-9140-47a7-87e5-8e091010034b service nova] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:07.317765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1165/2339] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:59:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:59:17.311956 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=28 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:59:20.419684 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c45988-c7f4-45e2-ba7a-64eed9f7c9de None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:20.518207 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c45988-c7f4-45e2-ba7a-64eed9f7c9de None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:20.523192 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a866d-24fe-4201-bf27-953efcd15dc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:20.523829 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a866d-24fe-4201-bf27-953efcd15dc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:20.524156 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a866d-24fe-4201-bf27-953efcd15dc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:20.524352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a866d-24fe-4201-bf27-953efcd15dc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:20.539307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a866d-24fe-4201-bf27-953efcd15dc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:20.542355 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a866d-24fe-4201-bf27-953efcd15dc7 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:20.563393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1175/2340] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:59:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:59:27.013111 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c883e-ea8c-48cf-bca2-4f2ebf2a95a6 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:27.013941 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c883e-ea8c-48cf-bca2-4f2ebf2a95a6 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:27.014401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c883e-ea8c-48cf-bca2-4f2ebf2a95a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:27.052418 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-dc8c883e-ea8c-48cf-bca2-4f2ebf2a95a6 None None] MFA Rules not processed for user `d6d78cbc233a498db3a9268a4b93d737`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:59:27.054663 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-dc8c883e-ea8c-48cf-bca2-4f2ebf2a95a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:27.068527 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1166/2341] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:59:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:59:30.554354 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=26 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:59:31.935654 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a866d-24fe-4201-bf27-953efcd15dc7 service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:31.945666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a866d-24fe-4201-bf27-953efcd15dc7 service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:31.949600 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719709c0-8af6-40f5-99ac-c379f8af0ce3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:31.950316 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719709c0-8af6-40f5-99ac-c379f8af0ce3 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:31.950590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719709c0-8af6-40f5-99ac-c379f8af0ce3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:31.950929 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719709c0-8af6-40f5-99ac-c379f8af0ce3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:31.952798 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-719709c0-8af6-40f5-99ac-c379f8af0ce3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:32.029919 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719709c0-8af6-40f5-99ac-c379f8af0ce3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:32.032839 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719709c0-8af6-40f5-99ac-c379f8af0ce3 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:32.051216 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1176/2342] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:59:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:59:37.042521 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:59:39.246469 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c883e-ea8c-48cf-bca2-4f2ebf2a95a6 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:39.247983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-dc8c883e-ea8c-48cf-bca2-4f2ebf2a95a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:39.387476 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c883e-ea8c-48cf-bca2-4f2ebf2a95a6 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:39.394137 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584be964-dc95-4703-aeb8-f917ae04d773 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e74dbb121d749749cf5f9d9d7221008', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:39.394814 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584be964-dc95-4703-aeb8-f917ae04d773 service placement] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:39.395090 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584be964-dc95-4703-aeb8-f917ae04d773 service placement] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:39.395356 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584be964-dc95-4703-aeb8-f917ae04d773 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:39.397588 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-584be964-dc95-4703-aeb8-f917ae04d773 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:39.413749 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584be964-dc95-4703-aeb8-f917ae04d773 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:39.416852 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584be964-dc95-4703-aeb8-f917ae04d773 service placement] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:39.438888 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1167/2343] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:59:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:59:39.495573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719709c0-8af6-40f5-99ac-c379f8af0ce3 service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:39.503188 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-719709c0-8af6-40f5-99ac-c379f8af0ce3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:39.521685 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719709c0-8af6-40f5-99ac-c379f8af0ce3 service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:39.526877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612f65e-57f3-467d-aaa3-b07e9847fda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:39.528541 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612f65e-57f3-467d-aaa3-b07e9847fda9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:39.529258 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612f65e-57f3-467d-aaa3-b07e9847fda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:39.529749 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612f65e-57f3-467d-aaa3-b07e9847fda9 admin admin] PATH_INFO: `/v3/users/75d793a42ec74232899b7a19e36b5867` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:39.531468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3612f65e-57f3-467d-aaa3-b07e9847fda9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75d793a42ec74232899b7a19e36b5867)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:39.534865 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3612f65e-57f3-467d-aaa3-b07e9847fda9 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:39.823337 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3612f65e-57f3-467d-aaa3-b07e9847fda9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75d793a42ec74232899b7a19e36b5867'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:39.937468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3612f65e-57f3-467d-aaa3-b07e9847fda9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75d793a42ec74232899b7a19e36b5867'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:39.945099 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3612f65e-57f3-467d-aaa3-b07e9847fda9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75d793a42ec74232899b7a19e36b5867'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:39.957932 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3612f65e-57f3-467d-aaa3-b07e9847fda9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75d793a42ec74232899b7a19e36b5867'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:39.991637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-3612f65e-57f3-467d-aaa3-b07e9847fda9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75d793a42ec74232899b7a19e36b5867'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:40.006831 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1177/2344] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:59:39 2022] DELETE /identity/v3/users/75d793a42ec74232899b7a19e36b5867 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:59:40.457041 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584be964-dc95-4703-aeb8-f917ae04d773 service placement] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:40.476194 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584be964-dc95-4703-aeb8-f917ae04d773 service placement] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:40.598830 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8139c-d8dc-41a6-b657-5e8ab9f15c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:40.600054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8139c-d8dc-41a6-b657-5e8ab9f15c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:40.600539 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8139c-d8dc-41a6-b657-5e8ab9f15c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:40.601022 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8139c-d8dc-41a6-b657-5e8ab9f15c7e admin admin] PATH_INFO: `/v3/projects/6cac5ca5a8b04bce9a9d8ac938cd35e8` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:40.603542 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf8139c-d8dc-41a6-b657-5e8ab9f15c7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cac5ca5a8b04bce9a9d8ac938cd35e8)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:40.607480 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf8139c-d8dc-41a6-b657-5e8ab9f15c7e admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:40.781597 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-bdf8139c-d8dc-41a6-b657-5e8ab9f15c7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cac5ca5a8b04bce9a9d8ac938cd35e8', 'request_id': 'req-bdf8139c-d8dc-41a6-b657-5e8ab9f15c7e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:40.794115 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-bdf8139c-d8dc-41a6-b657-5e8ab9f15c7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cac5ca5a8b04bce9a9d8ac938cd35e8', 'request_id': 'req-bdf8139c-d8dc-41a6-b657-5e8ab9f15c7e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:40.812061 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-bdf8139c-d8dc-41a6-b657-5e8ab9f15c7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cac5ca5a8b04bce9a9d8ac938cd35e8', 'request_id': 'req-bdf8139c-d8dc-41a6-b657-5e8ab9f15c7e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:40.921155 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-bdf8139c-d8dc-41a6-b657-5e8ab9f15c7e admin admin] The token cache is being invalidate because project 6cac5ca5a8b04bce9a9d8ac938cd35e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:59:40.921985 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-bdf8139c-d8dc-41a6-b657-5e8ab9f15c7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:40.923135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-bdf8139c-d8dc-41a6-b657-5e8ab9f15c7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:40.925798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1168/2345] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:59:40 2022] DELETE /identity/v3/projects/6cac5ca5a8b04bce9a9d8ac938cd35e8 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:59:40.933126 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f3090-e952-44e8-9355-9e4fdbef5d68 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:40.933721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f3090-e952-44e8-9355-9e4fdbef5d68 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:40.933721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f3090-e952-44e8-9355-9e4fdbef5d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:40.957959 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-eb6f3090-e952-44e8-9355-9e4fdbef5d68 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:59:40.959251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-eb6f3090-e952-44e8-9355-9e4fdbef5d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:41.034471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1178/2346] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:59:41.043437 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6e430-8b59-417e-88fb-5239b8e78033 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:41.043910 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6e430-8b59-417e-88fb-5239b8e78033 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:41.044458 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6e430-8b59-417e-88fb-5239b8e78033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:41.074137 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-e9d6e430-8b59-417e-88fb-5239b8e78033 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:59:41.075453 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-e9d6e430-8b59-417e-88fb-5239b8e78033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:41.086057 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1169/2347] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:59:41.091628 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f3090-e952-44e8-9355-9e4fdbef5d68 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:41.103923 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f3090-e952-44e8-9355-9e4fdbef5d68 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:41.110694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc52fc-05a9-4b92-aacd-7cf79ed8cbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:41.111324 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc52fc-05a9-4b92-aacd-7cf79ed8cbe4 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:41.111868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc52fc-05a9-4b92-aacd-7cf79ed8cbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:41.112533 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc52fc-05a9-4b92-aacd-7cf79ed8cbe4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:41.113220 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc52fc-05a9-4b92-aacd-7cf79ed8cbe4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:59:41.113757 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc52fc-05a9-4b92-aacd-7cf79ed8cbe4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:41.115467 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc52fc-05a9-4b92-aacd-7cf79ed8cbe4 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:41.118169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:59:41.130151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1179/2348] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:59:41.137391 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d6e430-8b59-417e-88fb-5239b8e78033 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:41.150232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d6e430-8b59-417e-88fb-5239b8e78033 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:41.155610 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614542d6-8ed6-42a9-9fde-eb2b852603f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:41.156239 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614542d6-8ed6-42a9-9fde-eb2b852603f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:41.156504 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614542d6-8ed6-42a9-9fde-eb2b852603f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:41.156772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614542d6-8ed6-42a9-9fde-eb2b852603f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:41.157466 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614542d6-8ed6-42a9-9fde-eb2b852603f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:41.160596 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614542d6-8ed6-42a9-9fde-eb2b852603f4 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:41.174474 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:59:41.307395 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1170/2349] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:59:41 2022] POST /identity/v3/projects => generated 367 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:59:41.309732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc52fc-05a9-4b92-aacd-7cf79ed8cbe4 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:41.340383 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc52fc-05a9-4b92-aacd-7cf79ed8cbe4 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:41.476182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b38f4-90e6-4a9b-8aab-7da281723e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:41.476923 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b38f4-90e6-4a9b-8aab-7da281723e99 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:41.477258 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b38f4-90e6-4a9b-8aab-7da281723e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:41.477602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b38f4-90e6-4a9b-8aab-7da281723e99 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:41.478350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b38f4-90e6-4a9b-8aab-7da281723e99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:41.480769 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b38f4-90e6-4a9b-8aab-7da281723e99 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:41.531883 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d0fb9-aa75-4d0a-88fb-9be710c24c03 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:41.532939 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d0fb9-aa75-4d0a-88fb-9be710c24c03 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:41.532939 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d0fb9-aa75-4d0a-88fb-9be710c24c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:41.574182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-3e9d0fb9-aa75-4d0a-88fb-9be710c24c03 None None] MFA Rules not processed for user `9646662b0b3048f79ad5d2b9602f10a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:59:41.576793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-3e9d0fb9-aa75-4d0a-88fb-9be710c24c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:41.624811 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1180/2350] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:59:41 2022] POST /identity/v3/users => generated 284 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:59:41.626750 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b38f4-90e6-4a9b-8aab-7da281723e99 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:41.647614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b38f4-90e6-4a9b-8aab-7da281723e99 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:41.655601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08a76a-06b9-4e91-aebb-c88f560fdcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:41.656422 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08a76a-06b9-4e91-aebb-c88f560fdcd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:41.661028 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08a76a-06b9-4e91-aebb-c88f560fdcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:41.661028 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08a76a-06b9-4e91-aebb-c88f560fdcd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:41.661028 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad08a76a-06b9-4e91-aebb-c88f560fdcd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:41.661503 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad08a76a-06b9-4e91-aebb-c88f560fdcd2 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:41.677004 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1181/2351] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:59:41.681246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08a76a-06b9-4e91-aebb-c88f560fdcd2 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:41.687597 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1171/2352] 10.222.0.13 () {58 vars in 967 bytes} [Mon Aug 8 16:59:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:59:41.696344 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08a76a-06b9-4e91-aebb-c88f560fdcd2 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:41.698973 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e02d76-0b80-4dc6-9b60-dfb0fc88e0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:41.699609 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e02d76-0b80-4dc6-9b60-dfb0fc88e0e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:41.699890 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e02d76-0b80-4dc6-9b60-dfb0fc88e0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:41.700180 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e02d76-0b80-4dc6-9b60-dfb0fc88e0e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:41.700990 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e02d76-0b80-4dc6-9b60-dfb0fc88e0e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:41.703049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e02d76-0b80-4dc6-9b60-dfb0fc88e0e4 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:41.715544 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1182/2353] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:59:41.720888 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d0fb9-aa75-4d0a-88fb-9be710c24c03 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:41.745954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d0fb9-aa75-4d0a-88fb-9be710c24c03 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:41.756172 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6a02c-4829-4377-b2db-ca73dc5f5b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:41.757228 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6a02c-4829-4377-b2db-ca73dc5f5b11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:41.758087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6a02c-4829-4377-b2db-ca73dc5f5b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:41.758535 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6a02c-4829-4377-b2db-ca73dc5f5b11 admin admin] PATH_INFO: `/v3/projects/25a9c915260c42f1919248b9cfef6053/users/cfd71beefa9d4d2fb7146b5a40dda38e/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:41.778868 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6a02c-4829-4377-b2db-ca73dc5f5b11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25a9c915260c42f1919248b9cfef6053, user_id=cfd71beefa9d4d2fb7146b5a40dda38e, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:41.785193 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6a02c-4829-4377-b2db-ca73dc5f5b11 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:41.921828 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e02d76-0b80-4dc6-9b60-dfb0fc88e0e4 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:41.926111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-31e02d76-0b80-4dc6-9b60-dfb0fc88e0e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:41.952416 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1172/2354] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:59:41 2022] PUT /identity/v3/projects/25a9c915260c42f1919248b9cfef6053/users/cfd71beefa9d4d2fb7146b5a40dda38e/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:59:41.972254 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e94e6-b45c-4ab5-8469-0dd9767ba14c None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:41.972697 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e94e6-b45c-4ab5-8469-0dd9767ba14c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:41.973132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e94e6-b45c-4ab5-8469-0dd9767ba14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:42.018204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-691e94e6-b45c-4ab5-8469-0dd9767ba14c None None] MFA Rules not processed for user `cfd71beefa9d4d2fb7146b5a40dda38e`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:59:42.022364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-691e94e6-b45c-4ab5-8469-0dd9767ba14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:42.067849 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e02d76-0b80-4dc6-9b60-dfb0fc88e0e4 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:42.169433 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1173/2355] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:59:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:59:42.177306 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043772d4-4512-418b-82b4-d9f6063e2518 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c606decb832444e38a194b98f8d15f8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:42.177973 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043772d4-4512-418b-82b4-d9f6063e2518 service swift] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:42.178354 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043772d4-4512-418b-82b4-d9f6063e2518 service swift] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:42.178776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043772d4-4512-418b-82b4-d9f6063e2518 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:42.183201 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e94e6-b45c-4ab5-8469-0dd9767ba14c None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:42.184288 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:59:42.185099 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-691e94e6-b45c-4ab5-8469-0dd9767ba14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:42.271704 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043772d4-4512-418b-82b4-d9f6063e2518 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:42.274088 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043772d4-4512-418b-82b4-d9f6063e2518 service swift] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:42.280033 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e94e6-b45c-4ab5-8469-0dd9767ba14c None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:42.286321 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2742b-7320-4c12-894d-ae2e8e22f1be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:42.287268 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2742b-7320-4c12-894d-ae2e8e22f1be service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:42.287710 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2742b-7320-4c12-894d-ae2e8e22f1be service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:42.288161 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2742b-7320-4c12-894d-ae2e8e22f1be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:42.291439 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1183/2356] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Aug 8 16:59:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 16:59:42.303157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2742b-7320-4c12-894d-ae2e8e22f1be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:42.305879 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2742b-7320-4c12-894d-ae2e8e22f1be service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:42.319953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1174/2357] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:59:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:59:47.955967 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043772d4-4512-418b-82b4-d9f6063e2518 service swift] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:47.958260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-043772d4-4512-418b-82b4-d9f6063e2518 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:48.037885 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea3197-4fc0-4545-bac6-f6152bddabee None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:48.039080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea3197-4fc0-4545-bac6-f6152bddabee None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:48.039080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea3197-4fc0-4545-bac6-f6152bddabee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:48.055640 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:59:48.073605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043772d4-4512-418b-82b4-d9f6063e2518 service swift] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:48.080637 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-cbea3197-4fc0-4545-bac6-f6152bddabee None None] MFA Rules not processed for user `fdb16e7edbe143e9b96325842896f04e`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:59:48.080883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470e843-9ea3-4c99-b6b9-0bc25f07ff6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:48.086731 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470e843-9ea3-4c99-b6b9-0bc25f07ff6f service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:48.086731 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470e843-9ea3-4c99-b6b9-0bc25f07ff6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:48.087604 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-cbea3197-4fc0-4545-bac6-f6152bddabee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:48.087720 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470e843-9ea3-4c99-b6b9-0bc25f07ff6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:48.087720 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f470e843-9ea3-4c99-b6b9-0bc25f07ff6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:48.191860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1175/2358] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:59:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:59:48.224675 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f470e843-9ea3-4c99-b6b9-0bc25f07ff6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:48.231389 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f470e843-9ea3-4c99-b6b9-0bc25f07ff6f service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:48.253452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1184/2359] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:59:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:59:54.764630 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea3197-4fc0-4545-bac6-f6152bddabee None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:54.882029 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea3197-4fc0-4545-bac6-f6152bddabee None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:54.888986 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991fa5e-2105-4295-b701-f4322d5ff0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:54.889863 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991fa5e-2105-4295-b701-f4322d5ff0ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:54.890194 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991fa5e-2105-4295-b701-f4322d5ff0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:54.890534 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991fa5e-2105-4295-b701-f4322d5ff0ec admin admin] PATH_INFO: `/v3/users/cfd71beefa9d4d2fb7146b5a40dda38e` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:54.892917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991fa5e-2105-4295-b701-f4322d5ff0ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfd71beefa9d4d2fb7146b5a40dda38e)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:54.895384 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991fa5e-2105-4295-b701-f4322d5ff0ec admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:55.190715 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-0991fa5e-2105-4295-b701-f4322d5ff0ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfd71beefa9d4d2fb7146b5a40dda38e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:55.316073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-0991fa5e-2105-4295-b701-f4322d5ff0ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfd71beefa9d4d2fb7146b5a40dda38e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:55.317539 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-0991fa5e-2105-4295-b701-f4322d5ff0ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfd71beefa9d4d2fb7146b5a40dda38e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:55.337428 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-0991fa5e-2105-4295-b701-f4322d5ff0ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfd71beefa9d4d2fb7146b5a40dda38e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:55.338594 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-0991fa5e-2105-4295-b701-f4322d5ff0ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfd71beefa9d4d2fb7146b5a40dda38e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:55.379532 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1176/2360] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:59:54 2022] DELETE /identity/v3/users/cfd71beefa9d4d2fb7146b5a40dda38e => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:59:55.384513 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbb914-d99f-4199-806d-4e2384a1ec6b None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:55.385338 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbb914-d99f-4199-806d-4e2384a1ec6b None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:55.385338 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbb914-d99f-4199-806d-4e2384a1ec6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:55.462904 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-9dbbb914-d99f-4199-806d-4e2384a1ec6b None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:59:55.465074 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-9dbbb914-d99f-4199-806d-4e2384a1ec6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:55.531066 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:59:55.615983 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1185/2361] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:59:55.641397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991fa5e-2105-4295-b701-f4322d5ff0ec admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:55.643454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-0991fa5e-2105-4295-b701-f4322d5ff0ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:55.756715 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:59:55.820026 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991fa5e-2105-4295-b701-f4322d5ff0ec admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:55.824911 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d35a5-95a3-4f1c-a20f-faaf53af5638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:55.826064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d35a5-95a3-4f1c-a20f-faaf53af5638 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:55.826399 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d35a5-95a3-4f1c-a20f-faaf53af5638 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:55.826704 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d35a5-95a3-4f1c-a20f-faaf53af5638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:55.847470 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016d35a5-95a3-4f1c-a20f-faaf53af5638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:55.849509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016d35a5-95a3-4f1c-a20f-faaf53af5638 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:55.863953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1177/2362] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 16:59:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:59:56.889630 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbb914-d99f-4199-806d-4e2384a1ec6b None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:56.895962 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-9dbbb914-d99f-4199-806d-4e2384a1ec6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:56.920028 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbb914-d99f-4199-806d-4e2384a1ec6b None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:56.929246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b0621-17d2-443a-9a5c-660d3f7782c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:56.930396 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b0621-17d2-443a-9a5c-660d3f7782c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:56.930911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b0621-17d2-443a-9a5c-660d3f7782c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:56.931660 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b0621-17d2-443a-9a5c-660d3f7782c9 admin admin] PATH_INFO: `/v3/projects/25a9c915260c42f1919248b9cfef6053` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:56.936251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b0621-17d2-443a-9a5c-660d3f7782c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25a9c915260c42f1919248b9cfef6053)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:56.944096 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b0621-17d2-443a-9a5c-660d3f7782c9 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:57.220021 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-a46b0621-17d2-443a-9a5c-660d3f7782c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25a9c915260c42f1919248b9cfef6053', 'request_id': 'req-a46b0621-17d2-443a-9a5c-660d3f7782c9'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:57.297686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-a46b0621-17d2-443a-9a5c-660d3f7782c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25a9c915260c42f1919248b9cfef6053', 'request_id': 'req-a46b0621-17d2-443a-9a5c-660d3f7782c9'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:57.306945 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-a46b0621-17d2-443a-9a5c-660d3f7782c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25a9c915260c42f1919248b9cfef6053', 'request_id': 'req-a46b0621-17d2-443a-9a5c-660d3f7782c9'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:57.320767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-a46b0621-17d2-443a-9a5c-660d3f7782c9 admin admin] The token cache is being invalidate because project 25a9c915260c42f1919248b9cfef6053 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:59:57.321273 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-a46b0621-17d2-443a-9a5c-660d3f7782c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:57.322145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-a46b0621-17d2-443a-9a5c-660d3f7782c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:57.325717 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1186/2363] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:59:56 2022] DELETE /identity/v3/projects/25a9c915260c42f1919248b9cfef6053 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:59:57.332061 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a5255-35db-4afa-b6b1-acf1e8220573 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:57.332958 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a5255-35db-4afa-b6b1-acf1e8220573 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:57.332958 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a5255-35db-4afa-b6b1-acf1e8220573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:57.409070 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-4a1a5255-35db-4afa-b6b1-acf1e8220573 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:59:57.410760 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-4a1a5255-35db-4afa-b6b1-acf1e8220573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:57.517894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1178/2364] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:59:57.526434 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1454aa-732f-41b5-bc49-824632352957 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:57.526761 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1454aa-732f-41b5-bc49-824632352957 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:57.527016 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1454aa-732f-41b5-bc49-824632352957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:57.554629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-9a1454aa-732f-41b5-bc49-824632352957 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:59:57.557583 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-9a1454aa-732f-41b5-bc49-824632352957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:57.576026 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a5255-35db-4afa-b6b1-acf1e8220573 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:57.577819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-4a1a5255-35db-4afa-b6b1-acf1e8220573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:57.587260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1187/2365] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:59:57.590133 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1454aa-732f-41b5-bc49-824632352957 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:57.599995 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a5255-35db-4afa-b6b1-acf1e8220573 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:57.601818 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1454aa-732f-41b5-bc49-824632352957 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:57.606857 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f14e0c-d5b9-4989-a231-ab7ed8469fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:57.607593 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f14e0c-d5b9-4989-a231-ab7ed8469fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:57.607772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f14e0c-d5b9-4989-a231-ab7ed8469fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:57.608028 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f14e0c-d5b9-4989-a231-ab7ed8469fc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:57.608700 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f14e0c-d5b9-4989-a231-ab7ed8469fc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:59:57.608954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b8503-3831-418b-92eb-2757818fd35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:57.609182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f14e0c-d5b9-4989-a231-ab7ed8469fc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:57.610214 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b8503-3831-418b-92eb-2757818fd35e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:57.611257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b8503-3831-418b-92eb-2757818fd35e admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:57.613046 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f14e0c-d5b9-4989-a231-ab7ed8469fc6 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:57.613230 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b8503-3831-418b-92eb-2757818fd35e admin admin] PATH_INFO: `/v3/users/fd667c5354f5499686dfe858bd159aa0` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:57.614267 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b8503-3831-418b-92eb-2757818fd35e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd667c5354f5499686dfe858bd159aa0)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:57.618087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b8503-3831-418b-92eb-2757818fd35e admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:57.630249 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1188/2366] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 16:59:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:59:57.631453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f14e0c-d5b9-4989-a231-ab7ed8469fc6 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:57.645094 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f14e0c-d5b9-4989-a231-ab7ed8469fc6 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:57.651109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b5a61-aadb-40b6-9894-efadb346e913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:57.652111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b5a61-aadb-40b6-9894-efadb346e913 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:57.652561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b5a61-aadb-40b6-9894-efadb346e913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:57.653016 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b5a61-aadb-40b6-9894-efadb346e913 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:57.654206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b5a61-aadb-40b6-9894-efadb346e913 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:57.658291 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b5a61-aadb-40b6-9894-efadb346e913 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:57.856033 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1189/2367] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:59:57 2022] POST /identity/v3/projects => generated 379 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:59:57.862667 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b5a61-aadb-40b6-9894-efadb346e913 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:57.903851 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-d15b8503-3831-418b-92eb-2757818fd35e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd667c5354f5499686dfe858bd159aa0'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:57.958372 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b5a61-aadb-40b6-9894-efadb346e913 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:57.966237 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c0dd1-ef27-4c43-bf88-f1209ae36e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:57.967270 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c0dd1-ef27-4c43-bf88-f1209ae36e3d admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:57.967745 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c0dd1-ef27-4c43-bf88-f1209ae36e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:57.968193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c0dd1-ef27-4c43-bf88-f1209ae36e3d admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:57.969422 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c0dd1-ef27-4c43-bf88-f1209ae36e3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:57.974028 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c0dd1-ef27-4c43-bf88-f1209ae36e3d admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:57.976832 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-d15b8503-3831-418b-92eb-2757818fd35e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd667c5354f5499686dfe858bd159aa0'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:57.984102 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-d15b8503-3831-418b-92eb-2757818fd35e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd667c5354f5499686dfe858bd159aa0'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:57.989371 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-d15b8503-3831-418b-92eb-2757818fd35e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd667c5354f5499686dfe858bd159aa0'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:57.990126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-d15b8503-3831-418b-92eb-2757818fd35e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd667c5354f5499686dfe858bd159aa0'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:58.016856 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1179/2368] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:59:57 2022] DELETE /identity/v3/users/fd667c5354f5499686dfe858bd159aa0 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:59:58.085846 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:59:58.092029 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1190/2369] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:59:57 2022] POST /identity/v3/users => generated 290 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:59:58.099010 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b8503-3831-418b-92eb-2757818fd35e admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:58.131326 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d15b8503-3831-418b-92eb-2757818fd35e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:58.146051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:59:58.226156 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b8503-3831-418b-92eb-2757818fd35e admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:58.232611 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2825fee-1f77-4ffc-b497-870811abccb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:58.233513 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2825fee-1f77-4ffc-b497-870811abccb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:58.233901 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2825fee-1f77-4ffc-b497-870811abccb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:58.234266 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2825fee-1f77-4ffc-b497-870811abccb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:58.235131 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2825fee-1f77-4ffc-b497-870811abccb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:58.238170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2825fee-1f77-4ffc-b497-870811abccb4 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:58.254020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1180/2370] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:59:58 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:59:58.259411 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c0dd1-ef27-4c43-bf88-f1209ae36e3d admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:58.276973 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c0dd1-ef27-4c43-bf88-f1209ae36e3d admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:58.285275 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c0ecc-a04e-4812-8daa-94d9087eab75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:58.286300 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c0ecc-a04e-4812-8daa-94d9087eab75 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:58.286828 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c0ecc-a04e-4812-8daa-94d9087eab75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:58.287364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c0ecc-a04e-4812-8daa-94d9087eab75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:58.288505 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c0ecc-a04e-4812-8daa-94d9087eab75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:58.291577 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c0ecc-a04e-4812-8daa-94d9087eab75 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:58.314624 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1191/2371] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:59:58 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:59:58.319941 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2825fee-1f77-4ffc-b497-870811abccb4 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:58.331028 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2825fee-1f77-4ffc-b497-870811abccb4 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:58.336298 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fdd475-c0da-4894-9ae4-d602cbe6eb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:58.337050 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdd475-c0da-4894-9ae4-d602cbe6eb80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:58.338721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdd475-c0da-4894-9ae4-d602cbe6eb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:58.338721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdd475-c0da-4894-9ae4-d602cbe6eb80 admin admin] PATH_INFO: `/v3/projects/9e8f893fbfa04f45a97da5ccf4f4624a/users/71e4f394828442ba9c0be1bf5bb5e41a/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:58.355996 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fdd475-c0da-4894-9ae4-d602cbe6eb80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e8f893fbfa04f45a97da5ccf4f4624a, user_id=71e4f394828442ba9c0be1bf5bb5e41a, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:58.359338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fdd475-c0da-4894-9ae4-d602cbe6eb80 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:58.371905 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c0ecc-a04e-4812-8daa-94d9087eab75 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:58.373922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-7c3c0ecc-a04e-4812-8daa-94d9087eab75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:58.388826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c0ecc-a04e-4812-8daa-94d9087eab75 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:58.396497 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589583d-f511-41d7-a9cd-17723acfd036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:58.398453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589583d-f511-41d7-a9cd-17723acfd036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:58.398990 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589583d-f511-41d7-a9cd-17723acfd036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:58.400340 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589583d-f511-41d7-a9cd-17723acfd036 admin admin] PATH_INFO: `/v3/projects/236c710547184e33b1b14a88402fb363` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:58.401650 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1589583d-f511-41d7-a9cd-17723acfd036 admin admin] RBAC: Authorizing `identity:delete_project(project_id=236c710547184e33b1b14a88402fb363)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:58.405517 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1589583d-f511-41d7-a9cd-17723acfd036 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:58.504058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1181/2372] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:59:58 2022] PUT /identity/v3/projects/9e8f893fbfa04f45a97da5ccf4f4624a/users/71e4f394828442ba9c0be1bf5bb5e41a/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:59:58.517901 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac668ee-1f1f-43bb-a780-73b2832b57df None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:58.518543 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac668ee-1f1f-43bb-a780-73b2832b57df None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:58.519000 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac668ee-1f1f-43bb-a780-73b2832b57df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:58.561492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-6ac668ee-1f1f-43bb-a780-73b2832b57df None None] MFA Rules not processed for user `71e4f394828442ba9c0be1bf5bb5e41a`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:59:58.562797 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-6ac668ee-1f1f-43bb-a780-73b2832b57df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:58.641213 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-1589583d-f511-41d7-a9cd-17723acfd036 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '236c710547184e33b1b14a88402fb363', 'request_id': 'req-1589583d-f511-41d7-a9cd-17723acfd036'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:58.658581 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1182/2373] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:59:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:59:58.663770 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac668ee-1f1f-43bb-a780-73b2832b57df None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:58.707939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-1589583d-f511-41d7-a9cd-17723acfd036 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '236c710547184e33b1b14a88402fb363', 'request_id': 'req-1589583d-f511-41d7-a9cd-17723acfd036'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:58.716258 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-1589583d-f511-41d7-a9cd-17723acfd036 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '236c710547184e33b1b14a88402fb363', 'request_id': 'req-1589583d-f511-41d7-a9cd-17723acfd036'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:58.730431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-1589583d-f511-41d7-a9cd-17723acfd036 admin admin] The token cache is being invalidate because project 236c710547184e33b1b14a88402fb363 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:59:58.731652 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-1589583d-f511-41d7-a9cd-17723acfd036 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:58.732560 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-1589583d-f511-41d7-a9cd-17723acfd036 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:58.734893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1192/2374] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:59:58 2022] DELETE /identity/v3/projects/236c710547184e33b1b14a88402fb363 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:59:58.744417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589583d-f511-41d7-a9cd-17723acfd036 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:58.756475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-1589583d-f511-41d7-a9cd-17723acfd036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:58.773695 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac668ee-1f1f-43bb-a780-73b2832b57df None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:58.822868 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:59:58.852248 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589583d-f511-41d7-a9cd-17723acfd036 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:58.858322 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31907f96-dbda-4e09-ab8f-71ae4d455548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:58.859022 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31907f96-dbda-4e09-ab8f-71ae4d455548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:58.859304 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31907f96-dbda-4e09-ab8f-71ae4d455548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:58.859593 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31907f96-dbda-4e09-ab8f-71ae4d455548 admin admin] PATH_INFO: `/v3/users/b66872cfc4634251bbc23574ad46d7b6` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:58.861046 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31907f96-dbda-4e09-ab8f-71ae4d455548 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b66872cfc4634251bbc23574ad46d7b6)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:58.863070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31907f96-dbda-4e09-ab8f-71ae4d455548 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:58.882664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b9855-ee44-4d4d-b977-cd657d52fdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:58.883312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b9855-ee44-4d4d-b977-cd657d52fdfa admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:58.883997 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b9855-ee44-4d4d-b977-cd657d52fdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:58.883997 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b9855-ee44-4d4d-b977-cd657d52fdfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:58.884480 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b9855-ee44-4d4d-b977-cd657d52fdfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:58.886473 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b9855-ee44-4d4d-b977-cd657d52fdfa admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:59.059621 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1183/2375] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 16:59:58 2022] POST /identity/v3/projects => generated 381 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:59:59.065330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b9855-ee44-4d4d-b977-cd657d52fdfa admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:59.067576 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-f41b9855-ee44-4d4d-b977-cd657d52fdfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:59.140456 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-31907f96-dbda-4e09-ab8f-71ae4d455548 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b66872cfc4634251bbc23574ad46d7b6'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:59.161131 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b9855-ee44-4d4d-b977-cd657d52fdfa admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:59.170419 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d789d3-b562-499d-9e1b-e8dd031e7eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:59.172577 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d789d3-b562-499d-9e1b-e8dd031e7eae admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:59.172577 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d789d3-b562-499d-9e1b-e8dd031e7eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:59.172577 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d789d3-b562-499d-9e1b-e8dd031e7eae admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:59.173680 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d789d3-b562-499d-9e1b-e8dd031e7eae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:59.175868 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d789d3-b562-499d-9e1b-e8dd031e7eae admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:59.239341 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-31907f96-dbda-4e09-ab8f-71ae4d455548 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b66872cfc4634251bbc23574ad46d7b6'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:59.239985 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-31907f96-dbda-4e09-ab8f-71ae4d455548 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b66872cfc4634251bbc23574ad46d7b6'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:59.248672 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-31907f96-dbda-4e09-ab8f-71ae4d455548 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b66872cfc4634251bbc23574ad46d7b6'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:59.258393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:59:59.263963 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-31907f96-dbda-4e09-ab8f-71ae4d455548 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b66872cfc4634251bbc23574ad46d7b6'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:59:59.274463 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1193/2376] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 16:59:58 2022] DELETE /identity/v3/users/b66872cfc4634251bbc23574ad46d7b6 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:59:59.281849 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1184/2377] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 16:59:59 2022] POST /identity/v3/users => generated 297 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:59:59.286516 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31907f96-dbda-4e09-ab8f-71ae4d455548 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:59.288066 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-31907f96-dbda-4e09-ab8f-71ae4d455548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:59.382465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31907f96-dbda-4e09-ab8f-71ae4d455548 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:59.390808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69f713-6380-4386-aebf-e2e53ad21d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:59.392725 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69f713-6380-4386-aebf-e2e53ad21d30 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:59.393234 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69f713-6380-4386-aebf-e2e53ad21d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:59.393747 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69f713-6380-4386-aebf-e2e53ad21d30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:59.395046 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69f713-6380-4386-aebf-e2e53ad21d30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:59.398360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69f713-6380-4386-aebf-e2e53ad21d30 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:59.421316 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1194/2378] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:59:59.426934 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d789d3-b562-499d-9e1b-e8dd031e7eae admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:59.444075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d789d3-b562-499d-9e1b-e8dd031e7eae admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:59.452274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd8ea8-0d34-4828-8468-839869e3df28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:59.453255 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd8ea8-0d34-4828-8468-839869e3df28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:59.453728 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd8ea8-0d34-4828-8468-839869e3df28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:59.454200 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd8ea8-0d34-4828-8468-839869e3df28 admin admin] PATH_INFO: `/v3/projects/83e2705b1a894731b10407ddab692d2a/users/a9486e61dc6e4a7888ba4f142b98696d/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:59.483060 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd8ea8-0d34-4828-8468-839869e3df28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83e2705b1a894731b10407ddab692d2a, user_id=a9486e61dc6e4a7888ba4f142b98696d, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:59.487625 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd8ea8-0d34-4828-8468-839869e3df28 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:59.691989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1185/2379] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:59:59 2022] PUT /identity/v3/projects/83e2705b1a894731b10407ddab692d2a/users/a9486e61dc6e4a7888ba4f142b98696d/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:59:59.696478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69f713-6380-4386-aebf-e2e53ad21d30 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:59.709208 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69f713-6380-4386-aebf-e2e53ad21d30 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:59.726355 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd8ea8-0d34-4828-8468-839869e3df28 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:59.729778 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-a0cd8ea8-0d34-4828-8468-839869e3df28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:59:59.827440 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05397d4f-b911-4db8-a097-6eae2effedc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:59.828141 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05397d4f-b911-4db8-a097-6eae2effedc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:59.828599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05397d4f-b911-4db8-a097-6eae2effedc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:59.828873 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05397d4f-b911-4db8-a097-6eae2effedc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:59.829563 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05397d4f-b911-4db8-a097-6eae2effedc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:59.831695 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05397d4f-b911-4db8-a097-6eae2effedc7 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:59.852000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1195/2380] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 16:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:59:59.856462 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05397d4f-b911-4db8-a097-6eae2effedc7 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:59:59.865172 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd8ea8-0d34-4828-8468-839869e3df28 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:59.868748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a229c-ce9a-4195-8f18-95f06751eaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:59.868942 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a229c-ce9a-4195-8f18-95f06751eaa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:59.869356 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a229c-ce9a-4195-8f18-95f06751eaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:59.869633 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a229c-ce9a-4195-8f18-95f06751eaa5 admin admin] PATH_INFO: `/v3/projects/8616d1e7e19b40c5a4317cf56d9ada30` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:59.871118 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529a229c-ce9a-4195-8f18-95f06751eaa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8616d1e7e19b40c5a4317cf56d9ada30)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:59.873922 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529a229c-ce9a-4195-8f18-95f06751eaa5 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:59.874798 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05397d4f-b911-4db8-a097-6eae2effedc7 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:59:59.882865 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea3d60-d299-4c4e-8e60-982748bfc586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:59:59.890492 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea3d60-d299-4c4e-8e60-982748bfc586 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:59.890955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea3d60-d299-4c4e-8e60-982748bfc586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:59.891185 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea3d60-d299-4c4e-8e60-982748bfc586 admin admin] PATH_INFO: `/v3/projects/83e2705b1a894731b10407ddab692d2a/users/a9486e61dc6e4a7888ba4f142b98696d/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:59.894078 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea3d60-d299-4c4e-8e60-982748bfc586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83e2705b1a894731b10407ddab692d2a, user_id=a9486e61dc6e4a7888ba4f142b98696d, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:59:59.899013 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea3d60-d299-4c4e-8e60-982748bfc586 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:59:59.911508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:59:59.917868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1196/2381] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 16:59:59 2022] PUT /identity/v3/projects/83e2705b1a894731b10407ddab692d2a/users/a9486e61dc6e4a7888ba4f142b98696d/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:59:59.929798 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29dce5-c061-4121-8cf3-6f3042330864 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:59:59.930777 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29dce5-c061-4121-8cf3-6f3042330864 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:59:59.931349 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29dce5-c061-4121-8cf3-6f3042330864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:59:59.970289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-4a29dce5-c061-4121-8cf3-6f3042330864 None None] MFA Rules not processed for user `a9486e61dc6e4a7888ba4f142b98696d`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:59:59.971969 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-4a29dce5-c061-4121-8cf3-6f3042330864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:00.050008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-529a229c-ce9a-4195-8f18-95f06751eaa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8616d1e7e19b40c5a4317cf56d9ada30', 'request_id': 'req-529a229c-ce9a-4195-8f18-95f06751eaa5'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:00.071391 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-529a229c-ce9a-4195-8f18-95f06751eaa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8616d1e7e19b40c5a4317cf56d9ada30', 'request_id': 'req-529a229c-ce9a-4195-8f18-95f06751eaa5'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:00.085701 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-529a229c-ce9a-4195-8f18-95f06751eaa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8616d1e7e19b40c5a4317cf56d9ada30', 'request_id': 'req-529a229c-ce9a-4195-8f18-95f06751eaa5'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:00.109964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:00.130354 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1197/2382] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 16:59:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:00.143658 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29dce5-c061-4121-8cf3-6f3042330864 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:00.145989 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-4a29dce5-c061-4121-8cf3-6f3042330864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:00.203500 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-529a229c-ce9a-4195-8f18-95f06751eaa5 admin admin] The token cache is being invalidate because project 8616d1e7e19b40c5a4317cf56d9ada30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:00.204015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-529a229c-ce9a-4195-8f18-95f06751eaa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:00.208447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-529a229c-ce9a-4195-8f18-95f06751eaa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:00.211678 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1186/2383] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 16:59:59 2022] DELETE /identity/v3/projects/8616d1e7e19b40c5a4317cf56d9ada30 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:00.221905 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29dce5-c061-4121-8cf3-6f3042330864 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:00.300730 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3401c-3eb5-42f6-bc3f-78fd2243da68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:00.300730 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3401c-3eb5-42f6-bc3f-78fd2243da68 service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:00.300730 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3401c-3eb5-42f6-bc3f-78fd2243da68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:00.300730 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3401c-3eb5-42f6-bc3f-78fd2243da68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:00.301210 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-02a3401c-3eb5-42f6-bc3f-78fd2243da68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:00.396219 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3401c-3eb5-42f6-bc3f-78fd2243da68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:00.399541 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3401c-3eb5-42f6-bc3f-78fd2243da68 service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:00.422245 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1198/2384] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:00 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:00.910607 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a229c-ce9a-4195-8f18-95f06751eaa5 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:00.912206 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-529a229c-ce9a-4195-8f18-95f06751eaa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:00.927497 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a229c-ce9a-4195-8f18-95f06751eaa5 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:00.931760 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108b26b-7b52-4901-a55a-c2179c2b359c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:00.932387 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108b26b-7b52-4901-a55a-c2179c2b359c service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:00.932683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108b26b-7b52-4901-a55a-c2179c2b359c service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:00.932910 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108b26b-7b52-4901-a55a-c2179c2b359c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:00.934874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-c108b26b-7b52-4901-a55a-c2179c2b359c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:01.021556 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108b26b-7b52-4901-a55a-c2179c2b359c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:01.025093 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108b26b-7b52-4901-a55a-c2179c2b359c service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:01.042480 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1187/2385] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:00:03.260740 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3401c-3eb5-42f6-bc3f-78fd2243da68 service cinder] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:03.263206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-02a3401c-3eb5-42f6-bc3f-78fd2243da68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:03.376391 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:03.417931 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3401c-3eb5-42f6-bc3f-78fd2243da68 service cinder] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:03.425405 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dd743-16c1-42e9-bb40-803acd416389 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e74dbb121d749749cf5f9d9d7221008', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:03.426166 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dd743-16c1-42e9-bb40-803acd416389 service placement] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:03.426501 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dd743-16c1-42e9-bb40-803acd416389 service placement] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:03.426895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dd743-16c1-42e9-bb40-803acd416389 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:03.429306 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-378dd743-16c1-42e9-bb40-803acd416389 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:03.444236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378dd743-16c1-42e9-bb40-803acd416389 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:03.446216 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378dd743-16c1-42e9-bb40-803acd416389 service placement] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:03.458537 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1199/2386] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:11.035237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:13.519641 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=13 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:15.027769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108b26b-7b52-4901-a55a-c2179c2b359c service cinder] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:15.029636 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-c108b26b-7b52-4901-a55a-c2179c2b359c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:15.108997 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108b26b-7b52-4901-a55a-c2179c2b359c service cinder] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:15.115196 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c942a10-f0d6-4448-9d48-10d3410e5579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:15.115851 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c942a10-f0d6-4448-9d48-10d3410e5579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:15.116117 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c942a10-f0d6-4448-9d48-10d3410e5579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:15.116380 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c942a10-f0d6-4448-9d48-10d3410e5579 admin admin] PATH_INFO: `/v3/users/71e4f394828442ba9c0be1bf5bb5e41a` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:15.117642 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c942a10-f0d6-4448-9d48-10d3410e5579 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71e4f394828442ba9c0be1bf5bb5e41a)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:15.119673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c942a10-f0d6-4448-9d48-10d3410e5579 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:15.325478 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-1c942a10-f0d6-4448-9d48-10d3410e5579 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71e4f394828442ba9c0be1bf5bb5e41a'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:15.398973 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-1c942a10-f0d6-4448-9d48-10d3410e5579 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71e4f394828442ba9c0be1bf5bb5e41a'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:15.399643 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-1c942a10-f0d6-4448-9d48-10d3410e5579 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71e4f394828442ba9c0be1bf5bb5e41a'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:15.409044 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-1c942a10-f0d6-4448-9d48-10d3410e5579 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71e4f394828442ba9c0be1bf5bb5e41a'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:15.411969 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-1c942a10-f0d6-4448-9d48-10d3410e5579 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71e4f394828442ba9c0be1bf5bb5e41a'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:15.430634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1188/2387] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:15 2022] DELETE /identity/v3/users/71e4f394828442ba9c0be1bf5bb5e41a => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:15.435859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e21b05-3ad7-4763-ab2e-6be30f08df67 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:15.436503 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e21b05-3ad7-4763-ab2e-6be30f08df67 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:15.436503 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e21b05-3ad7-4763-ab2e-6be30f08df67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:15.460331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-03e21b05-3ad7-4763-ab2e-6be30f08df67 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:15.461659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-03e21b05-3ad7-4763-ab2e-6be30f08df67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:15.533044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1200/2388] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:15.543974 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c942a10-f0d6-4448-9d48-10d3410e5579 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:15.545566 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-1c942a10-f0d6-4448-9d48-10d3410e5579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:15.643899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c942a10-f0d6-4448-9d48-10d3410e5579 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:15.647897 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133883d-3b9e-4752-b4e4-3ed0a1608d45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:15.648769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133883d-3b9e-4752-b4e4-3ed0a1608d45 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:15.648769 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133883d-3b9e-4752-b4e4-3ed0a1608d45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:15.649213 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133883d-3b9e-4752-b4e4-3ed0a1608d45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:15.661283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0133883d-3b9e-4752-b4e4-3ed0a1608d45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:15.663047 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0133883d-3b9e-4752-b4e4-3ed0a1608d45 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:15.675338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1189/2389] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:16.168779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e21b05-3ad7-4763-ab2e-6be30f08df67 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:16.171070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-03e21b05-3ad7-4763-ab2e-6be30f08df67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:16.194866 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e21b05-3ad7-4763-ab2e-6be30f08df67 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:16.201941 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6683f-b2a2-45cc-a1c2-efb3dda31fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:16.202956 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6683f-b2a2-45cc-a1c2-efb3dda31fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:16.203316 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6683f-b2a2-45cc-a1c2-efb3dda31fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:16.203582 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6683f-b2a2-45cc-a1c2-efb3dda31fb2 admin admin] PATH_INFO: `/v3/projects/9e8f893fbfa04f45a97da5ccf4f4624a` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:16.205766 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6683f-b2a2-45cc-a1c2-efb3dda31fb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e8f893fbfa04f45a97da5ccf4f4624a)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:16.207804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6683f-b2a2-45cc-a1c2-efb3dda31fb2 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:16.332301 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-41c6683f-b2a2-45cc-a1c2-efb3dda31fb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e8f893fbfa04f45a97da5ccf4f4624a', 'request_id': 'req-41c6683f-b2a2-45cc-a1c2-efb3dda31fb2'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:16.519163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-41c6683f-b2a2-45cc-a1c2-efb3dda31fb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e8f893fbfa04f45a97da5ccf4f4624a', 'request_id': 'req-41c6683f-b2a2-45cc-a1c2-efb3dda31fb2'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:16.527293 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-41c6683f-b2a2-45cc-a1c2-efb3dda31fb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e8f893fbfa04f45a97da5ccf4f4624a', 'request_id': 'req-41c6683f-b2a2-45cc-a1c2-efb3dda31fb2'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:16.539854 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-41c6683f-b2a2-45cc-a1c2-efb3dda31fb2 admin admin] The token cache is being invalidate because project 9e8f893fbfa04f45a97da5ccf4f4624a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:16.540187 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-41c6683f-b2a2-45cc-a1c2-efb3dda31fb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:16.540803 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-41c6683f-b2a2-45cc-a1c2-efb3dda31fb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:16.542336 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1201/2390] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:16 2022] DELETE /identity/v3/projects/9e8f893fbfa04f45a97da5ccf4f4624a => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:16.546778 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133883d-3b9e-4752-b4e4-3ed0a1608d45 service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:16.552762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:16.558773 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133883d-3b9e-4752-b4e4-3ed0a1608d45 service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:16.623057 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a624641-0242-4d36-b06e-9734ee5ba5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:16.624158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a624641-0242-4d36-b06e-9734ee5ba5ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:16.624437 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a624641-0242-4d36-b06e-9734ee5ba5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:16.624708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a624641-0242-4d36-b06e-9734ee5ba5ef admin admin] PATH_INFO: `/v3/users/a9486e61dc6e4a7888ba4f142b98696d` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:16.626286 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a624641-0242-4d36-b06e-9734ee5ba5ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9486e61dc6e4a7888ba4f142b98696d)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:16.628414 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a624641-0242-4d36-b06e-9734ee5ba5ef admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:16.899974 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-6a624641-0242-4d36-b06e-9734ee5ba5ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9486e61dc6e4a7888ba4f142b98696d'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:16.965581 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-6a624641-0242-4d36-b06e-9734ee5ba5ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9486e61dc6e4a7888ba4f142b98696d'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:16.966506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-6a624641-0242-4d36-b06e-9734ee5ba5ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9486e61dc6e4a7888ba4f142b98696d'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:16.980075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-6a624641-0242-4d36-b06e-9734ee5ba5ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9486e61dc6e4a7888ba4f142b98696d'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:16.980585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-6a624641-0242-4d36-b06e-9734ee5ba5ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9486e61dc6e4a7888ba4f142b98696d'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:17.005306 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1190/2391] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:16 2022] DELETE /identity/v3/users/a9486e61dc6e4a7888ba4f142b98696d => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:00:17.534767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6683f-b2a2-45cc-a1c2-efb3dda31fb2 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:17.535764 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-41c6683f-b2a2-45cc-a1c2-efb3dda31fb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:17.557199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6683f-b2a2-45cc-a1c2-efb3dda31fb2 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:17.635870 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57d0f1-d604-48c6-a929-465097773ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:17.636569 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57d0f1-d604-48c6-a929-465097773ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:17.636864 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57d0f1-d604-48c6-a929-465097773ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:17.637127 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57d0f1-d604-48c6-a929-465097773ed8 admin admin] PATH_INFO: `/v3/projects/83e2705b1a894731b10407ddab692d2a` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:17.638660 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57d0f1-d604-48c6-a929-465097773ed8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83e2705b1a894731b10407ddab692d2a)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:17.640632 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57d0f1-d604-48c6-a929-465097773ed8 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:17.761691 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:17.784785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0f57d0f1-d604-48c6-a929-465097773ed8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83e2705b1a894731b10407ddab692d2a', 'request_id': 'req-0f57d0f1-d604-48c6-a929-465097773ed8'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:17.873799 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0f57d0f1-d604-48c6-a929-465097773ed8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83e2705b1a894731b10407ddab692d2a', 'request_id': 'req-0f57d0f1-d604-48c6-a929-465097773ed8'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:17.882477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0f57d0f1-d604-48c6-a929-465097773ed8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83e2705b1a894731b10407ddab692d2a', 'request_id': 'req-0f57d0f1-d604-48c6-a929-465097773ed8'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:17.895773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0f57d0f1-d604-48c6-a929-465097773ed8 admin admin] The token cache is being invalidate because project 83e2705b1a894731b10407ddab692d2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:17.896060 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0f57d0f1-d604-48c6-a929-465097773ed8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:17.896806 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0f57d0f1-d604-48c6-a929-465097773ed8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:17.898395 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1202/2392] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:17 2022] DELETE /identity/v3/projects/83e2705b1a894731b10407ddab692d2a => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:17.904944 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34fe99-19db-4d5b-9108-2213d7c05f2e None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:17.905848 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34fe99-19db-4d5b-9108-2213d7c05f2e None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:17.905848 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34fe99-19db-4d5b-9108-2213d7c05f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:17.933790 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-8d34fe99-19db-4d5b-9108-2213d7c05f2e None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:17.936074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-8d34fe99-19db-4d5b-9108-2213d7c05f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:18.009057 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1191/2393] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:18.018118 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa0f03-1518-4e77-a830-38d90a26995c None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:18.018412 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa0f03-1518-4e77-a830-38d90a26995c None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:18.018857 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa0f03-1518-4e77-a830-38d90a26995c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:18.044213 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-f0aa0f03-1518-4e77-a830-38d90a26995c None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:18.044213 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f0aa0f03-1518-4e77-a830-38d90a26995c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:18.052505 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1203/2394] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:00:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:18.057476 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34fe99-19db-4d5b-9108-2213d7c05f2e None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:18.076999 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34fe99-19db-4d5b-9108-2213d7c05f2e None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:18.087025 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a5031-5b45-486e-ae11-847ece4ca6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:18.087817 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a5031-5b45-486e-ae11-847ece4ca6de admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:18.088361 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a5031-5b45-486e-ae11-847ece4ca6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:18.088686 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a5031-5b45-486e-ae11-847ece4ca6de admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:18.089417 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a5031-5b45-486e-ae11-847ece4ca6de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:00:18.089759 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a5031-5b45-486e-ae11-847ece4ca6de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:18.091550 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a5031-5b45-486e-ae11-847ece4ca6de admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:18.107207 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1192/2395] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 17:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:00:18.116023 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa0f03-1518-4e77-a830-38d90a26995c None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:18.133544 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa0f03-1518-4e77-a830-38d90a26995c None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:18.140472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424b2be-f241-412e-a02c-ea16596bd141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:18.141560 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424b2be-f241-412e-a02c-ea16596bd141 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:18.142043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424b2be-f241-412e-a02c-ea16596bd141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:18.142785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424b2be-f241-412e-a02c-ea16596bd141 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:18.143955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424b2be-f241-412e-a02c-ea16596bd141 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:18.147938 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424b2be-f241-412e-a02c-ea16596bd141 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:18.242555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1204/2396] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:18 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:18.247236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a5031-5b45-486e-ae11-847ece4ca6de admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:18.257783 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a5031-5b45-486e-ae11-847ece4ca6de admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:18.262813 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:18.319917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43278284-9800-40ea-bd50-1ba089e414fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:18.320440 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43278284-9800-40ea-bd50-1ba089e414fb admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:18.320713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43278284-9800-40ea-bd50-1ba089e414fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:18.320978 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43278284-9800-40ea-bd50-1ba089e414fb admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:18.321600 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43278284-9800-40ea-bd50-1ba089e414fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:18.323585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43278284-9800-40ea-bd50-1ba089e414fb admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:18.411531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1193/2397] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:18 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:18.416854 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424b2be-f241-412e-a02c-ea16596bd141 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:18.433257 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424b2be-f241-412e-a02c-ea16596bd141 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:18.439985 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa709cf2-baf2-4b81-bae3-44c7ab9c345e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:18.440621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa709cf2-baf2-4b81-bae3-44c7ab9c345e admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:18.440938 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa709cf2-baf2-4b81-bae3-44c7ab9c345e admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:18.441167 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa709cf2-baf2-4b81-bae3-44c7ab9c345e admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:18.444757 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa709cf2-baf2-4b81-bae3-44c7ab9c345e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:18.444757 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa709cf2-baf2-4b81-bae3-44c7ab9c345e admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:18.455953 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1205/2398] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:18.460397 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43278284-9800-40ea-bd50-1ba089e414fb admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:18.472389 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43278284-9800-40ea-bd50-1ba089e414fb admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:18.477440 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae5bd7c-7a0f-4e81-a5cb-2bb5f91c2504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:18.478055 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5bd7c-7a0f-4e81-a5cb-2bb5f91c2504 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:18.478335 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5bd7c-7a0f-4e81-a5cb-2bb5f91c2504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:18.478634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5bd7c-7a0f-4e81-a5cb-2bb5f91c2504 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:18.479287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5bd7c-7a0f-4e81-a5cb-2bb5f91c2504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:18.480920 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5bd7c-7a0f-4e81-a5cb-2bb5f91c2504 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:18.492818 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1194/2399] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:18.497356 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa709cf2-baf2-4b81-bae3-44c7ab9c345e admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:18.508746 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa709cf2-baf2-4b81-bae3-44c7ab9c345e admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:18.514119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483dffa-ec9a-46c0-9abc-8c792dfdd2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:18.515160 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483dffa-ec9a-46c0-9abc-8c792dfdd2d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:18.515547 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483dffa-ec9a-46c0-9abc-8c792dfdd2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:18.515868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483dffa-ec9a-46c0-9abc-8c792dfdd2d4 admin admin] PATH_INFO: `/v3/projects/217204c570854e0986aae3ffdb0eb6e0/users/3c156bdc1ddc46b296b1ff411a717612/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:18.533970 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5483dffa-ec9a-46c0-9abc-8c792dfdd2d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=217204c570854e0986aae3ffdb0eb6e0, user_id=3c156bdc1ddc46b296b1ff411a717612, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:18.536938 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5483dffa-ec9a-46c0-9abc-8c792dfdd2d4 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:18.628734 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1206/2400] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:18 2022] PUT /identity/v3/projects/217204c570854e0986aae3ffdb0eb6e0/users/3c156bdc1ddc46b296b1ff411a717612/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:18.643865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c04fde-bdc7-4b74-a11f-cace75fe5675 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:18.644153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c04fde-bdc7-4b74-a11f-cace75fe5675 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:18.644442 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c04fde-bdc7-4b74-a11f-cace75fe5675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:18.676649 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-b8c04fde-bdc7-4b74-a11f-cace75fe5675 None None] MFA Rules not processed for user `3c156bdc1ddc46b296b1ff411a717612`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:18.677894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-b8c04fde-bdc7-4b74-a11f-cace75fe5675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:18.753591 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1195/2401] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:00:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:18.758360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483dffa-ec9a-46c0-9abc-8c792dfdd2d4 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:18.826749 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483dffa-ec9a-46c0-9abc-8c792dfdd2d4 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:18.834408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32187f-5aa1-440a-ad82-0bf65a4421b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:18.834879 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32187f-5aa1-440a-ad82-0bf65a4421b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:18.835162 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32187f-5aa1-440a-ad82-0bf65a4421b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:18.835435 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32187f-5aa1-440a-ad82-0bf65a4421b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:18.836104 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32187f-5aa1-440a-ad82-0bf65a4421b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:18.838251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32187f-5aa1-440a-ad82-0bf65a4421b7 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:18.950779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1207/2402] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:18 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:00:18.957696 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c04fde-bdc7-4b74-a11f-cace75fe5675 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:18.971979 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c04fde-bdc7-4b74-a11f-cace75fe5675 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:19.058334 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a671d-6a8f-46cf-a822-f5b674ad7fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:19.059003 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a671d-6a8f-46cf-a822-f5b674ad7fed admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:19.059272 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a671d-6a8f-46cf-a822-f5b674ad7fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:19.059534 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a671d-6a8f-46cf-a822-f5b674ad7fed admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:19.060160 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a671d-6a8f-46cf-a822-f5b674ad7fed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:19.062310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a671d-6a8f-46cf-a822-f5b674ad7fed admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:19.218632 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1196/2403] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:18 2022] POST /identity/v3/users => generated 299 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:19.241727 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32187f-5aa1-440a-ad82-0bf65a4421b7 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:19.254329 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32187f-5aa1-440a-ad82-0bf65a4421b7 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:19.259375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b751aa1-43f6-4efd-b11d-a771e9061616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:19.259988 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b751aa1-43f6-4efd-b11d-a771e9061616 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:19.260262 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b751aa1-43f6-4efd-b11d-a771e9061616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:19.260525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b751aa1-43f6-4efd-b11d-a771e9061616 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:19.261173 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b751aa1-43f6-4efd-b11d-a771e9061616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:19.263028 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b751aa1-43f6-4efd-b11d-a771e9061616 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:19.277037 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1208/2404] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:00:19.281505 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a671d-6a8f-46cf-a822-f5b674ad7fed admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:19.295064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a671d-6a8f-46cf-a822-f5b674ad7fed admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:19.302255 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab585657-3194-4774-9099-d8bbd2fe804f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:19.303058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab585657-3194-4774-9099-d8bbd2fe804f admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:19.303533 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab585657-3194-4774-9099-d8bbd2fe804f admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:19.303663 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab585657-3194-4774-9099-d8bbd2fe804f admin admin] PATH_INFO: `/v3/projects/f7104ca6043e43f8b00a81f3a384aa6c/users/7e3e15eede004d1b85ef0812039cd15a/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:19.306448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:19.321592 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab585657-3194-4774-9099-d8bbd2fe804f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7104ca6043e43f8b00a81f3a384aa6c, user_id=7e3e15eede004d1b85ef0812039cd15a, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:19.324351 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab585657-3194-4774-9099-d8bbd2fe804f admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:19.429361 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1197/2405] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:19 2022] PUT /identity/v3/projects/f7104ca6043e43f8b00a81f3a384aa6c/users/7e3e15eede004d1b85ef0812039cd15a/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:19.434128 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b751aa1-43f6-4efd-b11d-a771e9061616 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:19.445964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b751aa1-43f6-4efd-b11d-a771e9061616 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:19.470956 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:19.515475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915466c-2f2c-4497-a34b-27dba1f537ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:19.516814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915466c-2f2c-4497-a34b-27dba1f537ea admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:19.517138 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915466c-2f2c-4497-a34b-27dba1f537ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:19.517557 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915466c-2f2c-4497-a34b-27dba1f537ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:19.518295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915466c-2f2c-4497-a34b-27dba1f537ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:19.520068 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915466c-2f2c-4497-a34b-27dba1f537ea admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:19.535521 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1209/2406] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:19.539803 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab585657-3194-4774-9099-d8bbd2fe804f admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:19.551026 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab585657-3194-4774-9099-d8bbd2fe804f admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:19.556176 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea178e34-cbe6-4688-acf3-a2579de372ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:19.556859 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea178e34-cbe6-4688-acf3-a2579de372ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:19.559708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea178e34-cbe6-4688-acf3-a2579de372ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:19.559708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea178e34-cbe6-4688-acf3-a2579de372ae admin admin] PATH_INFO: `/v3/projects/f7104ca6043e43f8b00a81f3a384aa6c/users/7e3e15eede004d1b85ef0812039cd15a/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:19.559708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea178e34-cbe6-4688-acf3-a2579de372ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7104ca6043e43f8b00a81f3a384aa6c, user_id=7e3e15eede004d1b85ef0812039cd15a, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:19.562266 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea178e34-cbe6-4688-acf3-a2579de372ae admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:19.577084 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1198/2407] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:19 2022] PUT /identity/v3/projects/f7104ca6043e43f8b00a81f3a384aa6c/users/7e3e15eede004d1b85ef0812039cd15a/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:19.589859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95c055-f213-462c-b613-5a9c11f6b868 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:19.594228 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95c055-f213-462c-b613-5a9c11f6b868 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:19.594228 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95c055-f213-462c-b613-5a9c11f6b868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:19.627605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-de95c055-f213-462c-b613-5a9c11f6b868 None None] MFA Rules not processed for user `7e3e15eede004d1b85ef0812039cd15a`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:19.628944 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-de95c055-f213-462c-b613-5a9c11f6b868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:19.730037 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1210/2408] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:00:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:19.744382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea178e34-cbe6-4688-acf3-a2579de372ae admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:19.746466 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ea178e34-cbe6-4688-acf3-a2579de372ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:19.857496 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea178e34-cbe6-4688-acf3-a2579de372ae admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:19.863790 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458d92a-f608-4c46-9c43-3d8b0d9d964d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:19.864060 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458d92a-f608-4c46-9c43-3d8b0d9d964d service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:19.864237 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458d92a-f608-4c46-9c43-3d8b0d9d964d service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:19.864561 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458d92a-f608-4c46-9c43-3d8b0d9d964d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:19.952166 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458d92a-f608-4c46-9c43-3d8b0d9d964d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:19.954555 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458d92a-f608-4c46-9c43-3d8b0d9d964d service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:19.968874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1199/2409] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:25.737640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95c055-f213-462c-b613-5a9c11f6b868 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:25.755692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95c055-f213-462c-b613-5a9c11f6b868 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:25.761601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23a7de-1416-411d-aa43-13176731304a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:25.762501 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23a7de-1416-411d-aa43-13176731304a service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:25.762967 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23a7de-1416-411d-aa43-13176731304a service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:25.763405 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23a7de-1416-411d-aa43-13176731304a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:25.780190 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23a7de-1416-411d-aa43-13176731304a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:25.782695 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23a7de-1416-411d-aa43-13176731304a service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:25.801029 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1211/2410] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:29.960806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=69,keystone:INSERT=2 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:32.433810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458d92a-f608-4c46-9c43-3d8b0d9d964d service cinder] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:32.507844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458d92a-f608-4c46-9c43-3d8b0d9d964d service cinder] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:32.513942 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a6eb1-40fe-4c29-a236-a7dc488d9a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:32.514706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a6eb1-40fe-4c29-a236-a7dc488d9a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:32.515053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a6eb1-40fe-4c29-a236-a7dc488d9a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:32.515415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a6eb1-40fe-4c29-a236-a7dc488d9a37 admin admin] PATH_INFO: `/v3/users/3c156bdc1ddc46b296b1ff411a717612` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:32.517732 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647a6eb1-40fe-4c29-a236-a7dc488d9a37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c156bdc1ddc46b296b1ff411a717612)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:32.519910 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647a6eb1-40fe-4c29-a236-a7dc488d9a37 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:32.769215 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-647a6eb1-40fe-4c29-a236-a7dc488d9a37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c156bdc1ddc46b296b1ff411a717612'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:32.850048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-647a6eb1-40fe-4c29-a236-a7dc488d9a37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c156bdc1ddc46b296b1ff411a717612'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:32.850774 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-647a6eb1-40fe-4c29-a236-a7dc488d9a37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c156bdc1ddc46b296b1ff411a717612'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:32.860163 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-647a6eb1-40fe-4c29-a236-a7dc488d9a37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c156bdc1ddc46b296b1ff411a717612'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:32.861415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-647a6eb1-40fe-4c29-a236-a7dc488d9a37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c156bdc1ddc46b296b1ff411a717612'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:32.883719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1200/2411] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:32 2022] DELETE /identity/v3/users/3c156bdc1ddc46b296b1ff411a717612 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:32.889497 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c9310-0ffa-473e-bca3-41591371a447 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:32.890093 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c9310-0ffa-473e-bca3-41591371a447 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:32.890473 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c9310-0ffa-473e-bca3-41591371a447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:32.916865 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-296c9310-0ffa-473e-bca3-41591371a447 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:32.918108 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-296c9310-0ffa-473e-bca3-41591371a447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:33.013948 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1212/2412] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:00:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:33.025325 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a6eb1-40fe-4c29-a236-a7dc488d9a37 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:33.026833 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-647a6eb1-40fe-4c29-a236-a7dc488d9a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:33.095282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a6eb1-40fe-4c29-a236-a7dc488d9a37 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:33.100454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc411dad-8c32-47e0-a29f-ad0db43ce522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:33.101080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc411dad-8c32-47e0-a29f-ad0db43ce522 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:33.101370 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc411dad-8c32-47e0-a29f-ad0db43ce522 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:33.101636 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc411dad-8c32-47e0-a29f-ad0db43ce522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:33.113337 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc411dad-8c32-47e0-a29f-ad0db43ce522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:33.115037 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc411dad-8c32-47e0-a29f-ad0db43ce522 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:33.126799 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1201/2413] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:33.721996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c9310-0ffa-473e-bca3-41591371a447 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:33.725170 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-296c9310-0ffa-473e-bca3-41591371a447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:33.747667 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c9310-0ffa-473e-bca3-41591371a447 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:33.755143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a56e11-8645-4d9c-8048-79b8986d33a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:33.756210 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a56e11-8645-4d9c-8048-79b8986d33a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:33.756750 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a56e11-8645-4d9c-8048-79b8986d33a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:33.757234 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a56e11-8645-4d9c-8048-79b8986d33a7 admin admin] PATH_INFO: `/v3/projects/217204c570854e0986aae3ffdb0eb6e0` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:33.759424 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a56e11-8645-4d9c-8048-79b8986d33a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=217204c570854e0986aae3ffdb0eb6e0)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:33.762916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a56e11-8645-4d9c-8048-79b8986d33a7 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:33.910440 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:33.931087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-30a56e11-8645-4d9c-8048-79b8986d33a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '217204c570854e0986aae3ffdb0eb6e0', 'request_id': 'req-30a56e11-8645-4d9c-8048-79b8986d33a7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:34.015500 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-30a56e11-8645-4d9c-8048-79b8986d33a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '217204c570854e0986aae3ffdb0eb6e0', 'request_id': 'req-30a56e11-8645-4d9c-8048-79b8986d33a7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:34.026486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-30a56e11-8645-4d9c-8048-79b8986d33a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '217204c570854e0986aae3ffdb0eb6e0', 'request_id': 'req-30a56e11-8645-4d9c-8048-79b8986d33a7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:34.041808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-30a56e11-8645-4d9c-8048-79b8986d33a7 admin admin] The token cache is being invalidate because project 217204c570854e0986aae3ffdb0eb6e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:34.042331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-30a56e11-8645-4d9c-8048-79b8986d33a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:34.043537 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-30a56e11-8645-4d9c-8048-79b8986d33a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:34.046246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1213/2414] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:33 2022] DELETE /identity/v3/projects/217204c570854e0986aae3ffdb0eb6e0 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:34.054457 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc411dad-8c32-47e0-a29f-ad0db43ce522 service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:34.070684 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc411dad-8c32-47e0-a29f-ad0db43ce522 service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:34.103906 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:34.155415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef23af-adf8-4efb-8e88-5dab655a27c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:34.156633 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef23af-adf8-4efb-8e88-5dab655a27c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:34.157077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef23af-adf8-4efb-8e88-5dab655a27c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:34.157423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef23af-adf8-4efb-8e88-5dab655a27c8 admin admin] PATH_INFO: `/v3/users/7e3e15eede004d1b85ef0812039cd15a` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:34.158767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef23af-adf8-4efb-8e88-5dab655a27c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e3e15eede004d1b85ef0812039cd15a)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:34.160802 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef23af-adf8-4efb-8e88-5dab655a27c8 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:34.443878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-6cef23af-adf8-4efb-8e88-5dab655a27c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e3e15eede004d1b85ef0812039cd15a'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:34.565391 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-6cef23af-adf8-4efb-8e88-5dab655a27c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e3e15eede004d1b85ef0812039cd15a'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:34.566126 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-6cef23af-adf8-4efb-8e88-5dab655a27c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e3e15eede004d1b85ef0812039cd15a'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:34.576524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-6cef23af-adf8-4efb-8e88-5dab655a27c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e3e15eede004d1b85ef0812039cd15a'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:34.577499 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-6cef23af-adf8-4efb-8e88-5dab655a27c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e3e15eede004d1b85ef0812039cd15a'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:34.603793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1202/2415] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:34 2022] DELETE /identity/v3/users/7e3e15eede004d1b85ef0812039cd15a => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:35.099636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a56e11-8645-4d9c-8048-79b8986d33a7 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:35.101323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-30a56e11-8645-4d9c-8048-79b8986d33a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:35.123434 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a56e11-8645-4d9c-8048-79b8986d33a7 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:35.197202 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef4ce9-5e9f-4edb-b267-c27cdf0f236a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:35.197867 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef4ce9-5e9f-4edb-b267-c27cdf0f236a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:35.198141 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef4ce9-5e9f-4edb-b267-c27cdf0f236a admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:35.198476 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef4ce9-5e9f-4edb-b267-c27cdf0f236a admin admin] PATH_INFO: `/v3/projects/f7104ca6043e43f8b00a81f3a384aa6c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:35.199939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef4ce9-5e9f-4edb-b267-c27cdf0f236a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7104ca6043e43f8b00a81f3a384aa6c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:35.202050 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef4ce9-5e9f-4edb-b267-c27cdf0f236a admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:35.363521 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-cdef4ce9-5e9f-4edb-b267-c27cdf0f236a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7104ca6043e43f8b00a81f3a384aa6c', 'request_id': 'req-cdef4ce9-5e9f-4edb-b267-c27cdf0f236a'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:35.462027 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-cdef4ce9-5e9f-4edb-b267-c27cdf0f236a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7104ca6043e43f8b00a81f3a384aa6c', 'request_id': 'req-cdef4ce9-5e9f-4edb-b267-c27cdf0f236a'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:35.475740 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-cdef4ce9-5e9f-4edb-b267-c27cdf0f236a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7104ca6043e43f8b00a81f3a384aa6c', 'request_id': 'req-cdef4ce9-5e9f-4edb-b267-c27cdf0f236a'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:35.494538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-cdef4ce9-5e9f-4edb-b267-c27cdf0f236a admin admin] The token cache is being invalidate because project f7104ca6043e43f8b00a81f3a384aa6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:35.494831 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-cdef4ce9-5e9f-4edb-b267-c27cdf0f236a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:35.495566 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-cdef4ce9-5e9f-4edb-b267-c27cdf0f236a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:35.497299 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1214/2416] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:35 2022] DELETE /identity/v3/projects/f7104ca6043e43f8b00a81f3a384aa6c => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:00:35.506943 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca03fdd-f6f9-4f20-a5be-042e674ed5cc None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:35.506943 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca03fdd-f6f9-4f20-a5be-042e674ed5cc None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:35.506943 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca03fdd-f6f9-4f20-a5be-042e674ed5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:35.528779 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-7ca03fdd-f6f9-4f20-a5be-042e674ed5cc None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:35.530008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-7ca03fdd-f6f9-4f20-a5be-042e674ed5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:35.644378 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1203/2417] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:35.655436 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b1d13-a41e-4fda-855a-7ccb1169797d None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:35.655436 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b1d13-a41e-4fda-855a-7ccb1169797d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:35.655436 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b1d13-a41e-4fda-855a-7ccb1169797d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:35.678685 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-647b1d13-a41e-4fda-855a-7ccb1169797d None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:35.679950 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-647b1d13-a41e-4fda-855a-7ccb1169797d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:35.690707 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1215/2418] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:35.695377 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca03fdd-f6f9-4f20-a5be-042e674ed5cc None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:35.707162 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca03fdd-f6f9-4f20-a5be-042e674ed5cc None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:35.712460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204a3d2-c461-4b4b-9501-6abe17f7b174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:35.713286 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204a3d2-c461-4b4b-9501-6abe17f7b174 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:35.713681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204a3d2-c461-4b4b-9501-6abe17f7b174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:35.713957 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204a3d2-c461-4b4b-9501-6abe17f7b174 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:35.714647 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e204a3d2-c461-4b4b-9501-6abe17f7b174 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:00:35.714934 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e204a3d2-c461-4b4b-9501-6abe17f7b174 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:35.716865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e204a3d2-c461-4b4b-9501-6abe17f7b174 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:35.730878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1204/2419] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 17:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:00:35.738561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b1d13-a41e-4fda-855a-7ccb1169797d None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:35.756810 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b1d13-a41e-4fda-855a-7ccb1169797d None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:35.764944 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e112293-6c96-4220-8264-d3642870145a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:35.766592 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e112293-6c96-4220-8264-d3642870145a admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:35.767082 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e112293-6c96-4220-8264-d3642870145a admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:35.767599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e112293-6c96-4220-8264-d3642870145a admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:35.768757 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e112293-6c96-4220-8264-d3642870145a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:35.772581 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e112293-6c96-4220-8264-d3642870145a admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:35.908340 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1216/2420] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:35 2022] POST /identity/v3/projects => generated 377 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:35.912084 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204a3d2-c461-4b4b-9501-6abe17f7b174 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:35.923755 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204a3d2-c461-4b4b-9501-6abe17f7b174 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:35.968195 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:35.986283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e112293-6c96-4220-8264-d3642870145a admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:35.988462 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-5e112293-6c96-4220-8264-d3642870145a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:36.014006 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:36.023066 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ef404-476c-4f1a-8c77-18e06adfc93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:36.023822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ef404-476c-4f1a-8c77-18e06adfc93c admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:36.024184 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ef404-476c-4f1a-8c77-18e06adfc93c admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:36.024526 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ef404-476c-4f1a-8c77-18e06adfc93c admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:36.025252 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ef404-476c-4f1a-8c77-18e06adfc93c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:36.028145 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ef404-476c-4f1a-8c77-18e06adfc93c admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:36.070690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e112293-6c96-4220-8264-d3642870145a admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:36.074833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351daf5-f90a-4c88-90d7-bc2bb78d05e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:36.075488 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351daf5-f90a-4c88-90d7-bc2bb78d05e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:36.075767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351daf5-f90a-4c88-90d7-bc2bb78d05e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:36.076039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351daf5-f90a-4c88-90d7-bc2bb78d05e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:36.078903 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-6351daf5-f90a-4c88-90d7-bc2bb78d05e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:36.092737 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351daf5-f90a-4c88-90d7-bc2bb78d05e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:36.094525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351daf5-f90a-4c88-90d7-bc2bb78d05e2 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:36.106660 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1217/2421] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:36.156729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1205/2422] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:35 2022] POST /identity/v3/users => generated 289 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:00:36.161430 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351daf5-f90a-4c88-90d7-bc2bb78d05e2 service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:36.175193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351daf5-f90a-4c88-90d7-bc2bb78d05e2 service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:36.181570 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac35445-3374-4ca5-9057-4899d0960194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:36.182445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac35445-3374-4ca5-9057-4899d0960194 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:36.182985 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac35445-3374-4ca5-9057-4899d0960194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:36.183431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac35445-3374-4ca5-9057-4899d0960194 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:36.184480 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac35445-3374-4ca5-9057-4899d0960194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:36.186990 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac35445-3374-4ca5-9057-4899d0960194 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:36.207486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1218/2423] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:36.208772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ef404-476c-4f1a-8c77-18e06adfc93c admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:36.220057 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ef404-476c-4f1a-8c77-18e06adfc93c admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:36.225085 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a717b-73f7-4e7e-a191-426f3c74c2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:36.225713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a717b-73f7-4e7e-a191-426f3c74c2d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:36.225989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a717b-73f7-4e7e-a191-426f3c74c2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:36.226256 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a717b-73f7-4e7e-a191-426f3c74c2d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:36.226947 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a717b-73f7-4e7e-a191-426f3c74c2d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:36.228640 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a717b-73f7-4e7e-a191-426f3c74c2d1 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:36.241208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1206/2424] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:36.246416 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac35445-3374-4ca5-9057-4899d0960194 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:36.259229 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac35445-3374-4ca5-9057-4899d0960194 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:36.264272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af156c68-709a-4c9b-a7bc-a45a58f1571f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:36.266139 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af156c68-709a-4c9b-a7bc-a45a58f1571f admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:36.266491 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af156c68-709a-4c9b-a7bc-a45a58f1571f admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:36.266743 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af156c68-709a-4c9b-a7bc-a45a58f1571f admin admin] PATH_INFO: `/v3/projects/6edade7c5d3e42ba97a28e0e2bf52d26/users/a7e1b4d982e348428e97d8d48946f2be/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:36.287391 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af156c68-709a-4c9b-a7bc-a45a58f1571f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6edade7c5d3e42ba97a28e0e2bf52d26, user_id=a7e1b4d982e348428e97d8d48946f2be, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:36.292913 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af156c68-709a-4c9b-a7bc-a45a58f1571f admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:36.409769 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1219/2425] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:36 2022] PUT /identity/v3/projects/6edade7c5d3e42ba97a28e0e2bf52d26/users/a7e1b4d982e348428e97d8d48946f2be/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:36.430160 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b80ca-288a-4202-93b5-b21340f6bf85 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:36.430680 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b80ca-288a-4202-93b5-b21340f6bf85 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:36.431155 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b80ca-288a-4202-93b5-b21340f6bf85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:36.471810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-b24b80ca-288a-4202-93b5-b21340f6bf85 None None] MFA Rules not processed for user `a7e1b4d982e348428e97d8d48946f2be`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:36.474489 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-b24b80ca-288a-4202-93b5-b21340f6bf85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:36.576089 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1207/2426] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:00:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:36.590237 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af156c68-709a-4c9b-a7bc-a45a58f1571f admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:36.597070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-af156c68-709a-4c9b-a7bc-a45a58f1571f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:36.706918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af156c68-709a-4c9b-a7bc-a45a58f1571f admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:36.714188 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aeb03b-0ceb-44e4-9860-75623814a450 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea51ef02128a4b63b11049eb0daef298', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:36.715102 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aeb03b-0ceb-44e4-9860-75623814a450 service glance] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:36.715377 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aeb03b-0ceb-44e4-9860-75623814a450 service glance] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:36.715644 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aeb03b-0ceb-44e4-9860-75623814a450 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:36.726371 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aeb03b-0ceb-44e4-9860-75623814a450 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:36.727890 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aeb03b-0ceb-44e4-9860-75623814a450 service glance] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:36.739177 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1220/2427] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:41.271340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b80ca-288a-4202-93b5-b21340f6bf85 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:41.273453 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-b24b80ca-288a-4202-93b5-b21340f6bf85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:41.389225 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b80ca-288a-4202-93b5-b21340f6bf85 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:41.394921 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ef298-ed63-4f53-853e-b61b1dccda78 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e74dbb121d749749cf5f9d9d7221008', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:41.395651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ef298-ed63-4f53-853e-b61b1dccda78 service placement] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:41.395927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ef298-ed63-4f53-853e-b61b1dccda78 service placement] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:41.396194 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ef298-ed63-4f53-853e-b61b1dccda78 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:41.398112 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-cc3ef298-ed63-4f53-853e-b61b1dccda78 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:41.412414 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ef298-ed63-4f53-853e-b61b1dccda78 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:41.414634 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ef298-ed63-4f53-853e-b61b1dccda78 service placement] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:41.428114 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1208/2428] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:41.456399 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aeb03b-0ceb-44e4-9860-75623814a450 service glance] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:41.458501 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f1aeb03b-0ceb-44e4-9860-75623814a450 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:41.544827 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aeb03b-0ceb-44e4-9860-75623814a450 service glance] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:41.551141 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbe910-1267-4637-8f11-091b3fb7a02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:41.551818 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbe910-1267-4637-8f11-091b3fb7a02c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:41.552163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbe910-1267-4637-8f11-091b3fb7a02c admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:41.552466 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbe910-1267-4637-8f11-091b3fb7a02c admin admin] PATH_INFO: `/v3/users/5a8db3ff64ca44958fd633ba25ee191f` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:41.553724 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbe910-1267-4637-8f11-091b3fb7a02c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a8db3ff64ca44958fd633ba25ee191f)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:41.555740 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbe910-1267-4637-8f11-091b3fb7a02c admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:41.708036 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:41.726156 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-66bbe910-1267-4637-8f11-091b3fb7a02c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a8db3ff64ca44958fd633ba25ee191f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:41.793161 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-66bbe910-1267-4637-8f11-091b3fb7a02c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a8db3ff64ca44958fd633ba25ee191f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:41.793887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-66bbe910-1267-4637-8f11-091b3fb7a02c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a8db3ff64ca44958fd633ba25ee191f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:41.802986 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-66bbe910-1267-4637-8f11-091b3fb7a02c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a8db3ff64ca44958fd633ba25ee191f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:41.817674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-66bbe910-1267-4637-8f11-091b3fb7a02c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a8db3ff64ca44958fd633ba25ee191f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:41.829449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1221/2429] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:41 2022] DELETE /identity/v3/users/5a8db3ff64ca44958fd633ba25ee191f => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:42.187573 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ef298-ed63-4f53-853e-b61b1dccda78 service placement] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:42.209474 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ef298-ed63-4f53-853e-b61b1dccda78 service placement] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:42.246236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:42.287486 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30f31c-6f86-4231-b6d1-b8dde5591daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:42.288539 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30f31c-6f86-4231-b6d1-b8dde5591daf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:42.288885 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30f31c-6f86-4231-b6d1-b8dde5591daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:42.289200 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30f31c-6f86-4231-b6d1-b8dde5591daf admin admin] PATH_INFO: `/v3/projects/442f2ca88fce4cbab9c01278239b77ce` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:42.290595 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b30f31c-6f86-4231-b6d1-b8dde5591daf admin admin] RBAC: Authorizing `identity:delete_project(project_id=442f2ca88fce4cbab9c01278239b77ce)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:42.292809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b30f31c-6f86-4231-b6d1-b8dde5591daf admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:42.451273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-0b30f31c-6f86-4231-b6d1-b8dde5591daf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '442f2ca88fce4cbab9c01278239b77ce', 'request_id': 'req-0b30f31c-6f86-4231-b6d1-b8dde5591daf'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:42.467017 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-0b30f31c-6f86-4231-b6d1-b8dde5591daf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '442f2ca88fce4cbab9c01278239b77ce', 'request_id': 'req-0b30f31c-6f86-4231-b6d1-b8dde5591daf'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:42.486301 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-0b30f31c-6f86-4231-b6d1-b8dde5591daf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '442f2ca88fce4cbab9c01278239b77ce', 'request_id': 'req-0b30f31c-6f86-4231-b6d1-b8dde5591daf'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:42.586039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-0b30f31c-6f86-4231-b6d1-b8dde5591daf admin admin] The token cache is being invalidate because project 442f2ca88fce4cbab9c01278239b77ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:42.587134 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-0b30f31c-6f86-4231-b6d1-b8dde5591daf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:42.588786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-0b30f31c-6f86-4231-b6d1-b8dde5591daf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:42.591015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1209/2430] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:42 2022] DELETE /identity/v3/projects/442f2ca88fce4cbab9c01278239b77ce => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:42.598008 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad90eb-62d7-4b10-9fc0-9f6f3cccbf08 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:42.598955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad90eb-62d7-4b10-9fc0-9f6f3cccbf08 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:42.598955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad90eb-62d7-4b10-9fc0-9f6f3cccbf08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:42.620708 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-f1ad90eb-62d7-4b10-9fc0-9f6f3cccbf08 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:42.621946 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f1ad90eb-62d7-4b10-9fc0-9f6f3cccbf08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:42.691270 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1222/2431] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:42.699824 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307684c-0a0a-4ecb-92d4-1b6348676c31 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:42.700187 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307684c-0a0a-4ecb-92d4-1b6348676c31 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:42.700510 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307684c-0a0a-4ecb-92d4-1b6348676c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:42.723193 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-5307684c-0a0a-4ecb-92d4-1b6348676c31 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:42.724444 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-5307684c-0a0a-4ecb-92d4-1b6348676c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:42.732651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1210/2432] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:00:42.737380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad90eb-62d7-4b10-9fc0-9f6f3cccbf08 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:42.747526 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad90eb-62d7-4b10-9fc0-9f6f3cccbf08 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:42.753126 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed8c49-36b2-4799-acb4-83aec5b0227d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:42.753749 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed8c49-36b2-4799-acb4-83aec5b0227d admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:42.754028 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed8c49-36b2-4799-acb4-83aec5b0227d admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:42.754298 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed8c49-36b2-4799-acb4-83aec5b0227d admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:42.754997 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed8c49-36b2-4799-acb4-83aec5b0227d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:00:42.755299 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed8c49-36b2-4799-acb4-83aec5b0227d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:42.756923 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed8c49-36b2-4799-acb4-83aec5b0227d admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:42.775622 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1223/2433] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 17:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:00:42.780790 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307684c-0a0a-4ecb-92d4-1b6348676c31 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:42.793978 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307684c-0a0a-4ecb-92d4-1b6348676c31 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:42.799129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f53434-8bfb-484f-a19f-1206ebe6829d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:42.799773 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f53434-8bfb-484f-a19f-1206ebe6829d admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:42.800049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f53434-8bfb-484f-a19f-1206ebe6829d admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:42.800318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f53434-8bfb-484f-a19f-1206ebe6829d admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:42.800960 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f53434-8bfb-484f-a19f-1206ebe6829d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:42.802930 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f53434-8bfb-484f-a19f-1206ebe6829d admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:42.924990 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1211/2434] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:42 2022] POST /identity/v3/projects => generated 363 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:42.929085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed8c49-36b2-4799-acb4-83aec5b0227d admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:42.941592 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed8c49-36b2-4799-acb4-83aec5b0227d admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:43.014964 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f841a-11cd-4d20-aab4-00d8ca17fb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:43.016326 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f841a-11cd-4d20-aab4-00d8ca17fb61 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:43.016602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f841a-11cd-4d20-aab4-00d8ca17fb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:43.016876 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f841a-11cd-4d20-aab4-00d8ca17fb61 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:43.017554 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f841a-11cd-4d20-aab4-00d8ca17fb61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:43.019579 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f841a-11cd-4d20-aab4-00d8ca17fb61 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:43.153331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1224/2435] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:42 2022] POST /identity/v3/users => generated 282 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:00:43.158854 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f53434-8bfb-484f-a19f-1206ebe6829d admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:43.169661 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f53434-8bfb-484f-a19f-1206ebe6829d admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:43.174509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccdf69-9e2f-43e6-847e-b7c3946c336f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:43.175157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccdf69-9e2f-43e6-847e-b7c3946c336f admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:43.175429 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccdf69-9e2f-43e6-847e-b7c3946c336f admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:43.175692 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccdf69-9e2f-43e6-847e-b7c3946c336f admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:43.176338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccdf69-9e2f-43e6-847e-b7c3946c336f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:43.178161 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccdf69-9e2f-43e6-847e-b7c3946c336f admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:43.189855 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1212/2436] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:43.195048 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f841a-11cd-4d20-aab4-00d8ca17fb61 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:43.211248 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f841a-11cd-4d20-aab4-00d8ca17fb61 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:43.218809 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7dbcaf-e1c6-40b0-95b6-b9df696b518e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:43.219533 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7dbcaf-e1c6-40b0-95b6-b9df696b518e admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:43.219811 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7dbcaf-e1c6-40b0-95b6-b9df696b518e admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:43.220076 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7dbcaf-e1c6-40b0-95b6-b9df696b518e admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:43.220731 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7dbcaf-e1c6-40b0-95b6-b9df696b518e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:43.222324 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7dbcaf-e1c6-40b0-95b6-b9df696b518e admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:43.235826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1225/2437] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:43.241633 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccdf69-9e2f-43e6-847e-b7c3946c336f admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:43.254378 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccdf69-9e2f-43e6-847e-b7c3946c336f admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:43.260148 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5554ff-242d-4a1b-8c47-68bab8b8443f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:43.261258 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5554ff-242d-4a1b-8c47-68bab8b8443f admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:43.261542 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5554ff-242d-4a1b-8c47-68bab8b8443f admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:43.261847 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5554ff-242d-4a1b-8c47-68bab8b8443f admin admin] PATH_INFO: `/v3/projects/3323ecb089054f5491b4f93669194b7a/users/0270311c3b6c4a2b8534fcaefc1c95c6/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:43.282159 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5554ff-242d-4a1b-8c47-68bab8b8443f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3323ecb089054f5491b4f93669194b7a, user_id=0270311c3b6c4a2b8534fcaefc1c95c6, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:43.284992 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5554ff-242d-4a1b-8c47-68bab8b8443f admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:43.389228 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1213/2438] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:43 2022] PUT /identity/v3/projects/3323ecb089054f5491b4f93669194b7a/users/0270311c3b6c4a2b8534fcaefc1c95c6/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:43.408358 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83f500-2fe5-44c4-b62f-b79660cfa10f None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:43.408843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83f500-2fe5-44c4-b62f-b79660cfa10f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:43.409335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83f500-2fe5-44c4-b62f-b79660cfa10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:43.445723 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:43.449608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-8e83f500-2fe5-44c4-b62f-b79660cfa10f None None] MFA Rules not processed for user `0270311c3b6c4a2b8534fcaefc1c95c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:43.451113 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-8e83f500-2fe5-44c4-b62f-b79660cfa10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:43.538143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1226/2439] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:00:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:43.543348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5554ff-242d-4a1b-8c47-68bab8b8443f admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:43.615025 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:43.626451 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5554ff-242d-4a1b-8c47-68bab8b8443f admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:43.631402 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82070154-ed0f-4837-91b3-84946427de59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:43.632052 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82070154-ed0f-4837-91b3-84946427de59 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:43.632334 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82070154-ed0f-4837-91b3-84946427de59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:43.632609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82070154-ed0f-4837-91b3-84946427de59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:43.633256 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82070154-ed0f-4837-91b3-84946427de59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:43.635292 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82070154-ed0f-4837-91b3-84946427de59 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:43.723873 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1214/2440] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:43 2022] POST /identity/v3/projects => generated 365 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:43.728594 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83f500-2fe5-44c4-b62f-b79660cfa10f None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:43.741785 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83f500-2fe5-44c4-b62f-b79660cfa10f None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:43.837334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527efb97-0d45-48ba-b8c4-f8765b02a57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:43.837334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527efb97-0d45-48ba-b8c4-f8765b02a57d admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:43.837334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527efb97-0d45-48ba-b8c4-f8765b02a57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:43.837334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527efb97-0d45-48ba-b8c4-f8765b02a57d admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:43.838147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527efb97-0d45-48ba-b8c4-f8765b02a57d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:43.838461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527efb97-0d45-48ba-b8c4-f8765b02a57d admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:44.021626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1227/2441] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:43 2022] POST /identity/v3/users => generated 289 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:44.026302 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82070154-ed0f-4837-91b3-84946427de59 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:44.040921 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82070154-ed0f-4837-91b3-84946427de59 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:44.049420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8903bba-d791-4a56-9a59-e59bcafa621e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:44.050445 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8903bba-d791-4a56-9a59-e59bcafa621e admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:44.050936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8903bba-d791-4a56-9a59-e59bcafa621e admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:44.051558 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8903bba-d791-4a56-9a59-e59bcafa621e admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:44.052905 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8903bba-d791-4a56-9a59-e59bcafa621e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:44.056048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8903bba-d791-4a56-9a59-e59bcafa621e admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:44.076892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1215/2442] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:44.082672 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527efb97-0d45-48ba-b8c4-f8765b02a57d admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:44.094721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527efb97-0d45-48ba-b8c4-f8765b02a57d admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:44.099685 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153bac08-0a42-4930-9c4e-5432ef400a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:44.100379 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153bac08-0a42-4930-9c4e-5432ef400a78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:44.100660 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153bac08-0a42-4930-9c4e-5432ef400a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:44.100935 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153bac08-0a42-4930-9c4e-5432ef400a78 admin admin] PATH_INFO: `/v3/projects/ee8ab01efcef49d7a9df6f67af2638fe/users/70b57718e8fa417386db06fd42a0fde2/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:44.119056 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153bac08-0a42-4930-9c4e-5432ef400a78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee8ab01efcef49d7a9df6f67af2638fe, user_id=70b57718e8fa417386db06fd42a0fde2, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:44.121895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153bac08-0a42-4930-9c4e-5432ef400a78 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:44.228415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1228/2443] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:44 2022] PUT /identity/v3/projects/ee8ab01efcef49d7a9df6f67af2638fe/users/70b57718e8fa417386db06fd42a0fde2/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:00:44.233169 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8903bba-d791-4a56-9a59-e59bcafa621e admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:44.244535 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8903bba-d791-4a56-9a59-e59bcafa621e admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:44.322113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a767c-8049-4a09-a356-1210117eb3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:44.322771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a767c-8049-4a09-a356-1210117eb3aa admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:44.323052 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a767c-8049-4a09-a356-1210117eb3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:44.323329 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a767c-8049-4a09-a356-1210117eb3aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:44.323979 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a767c-8049-4a09-a356-1210117eb3aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:44.325611 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a767c-8049-4a09-a356-1210117eb3aa admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:44.337480 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1216/2444] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:44.341994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153bac08-0a42-4930-9c4e-5432ef400a78 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:44.364963 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153bac08-0a42-4930-9c4e-5432ef400a78 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:44.364963 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c677f2-ae2e-42ce-a16d-0e609c37ca60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:44.364963 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c677f2-ae2e-42ce-a16d-0e609c37ca60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:44.364963 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c677f2-ae2e-42ce-a16d-0e609c37ca60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:44.365796 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c677f2-ae2e-42ce-a16d-0e609c37ca60 admin admin] PATH_INFO: `/v3/projects/ee8ab01efcef49d7a9df6f67af2638fe/users/70b57718e8fa417386db06fd42a0fde2/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:44.365796 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c677f2-ae2e-42ce-a16d-0e609c37ca60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee8ab01efcef49d7a9df6f67af2638fe, user_id=70b57718e8fa417386db06fd42a0fde2, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:44.367740 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c677f2-ae2e-42ce-a16d-0e609c37ca60 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:44.383687 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1229/2445] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:44 2022] PUT /identity/v3/projects/ee8ab01efcef49d7a9df6f67af2638fe/users/70b57718e8fa417386db06fd42a0fde2/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:00:44.404214 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18133bde-5626-4516-b126-07b620c0d639 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:44.404826 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18133bde-5626-4516-b126-07b620c0d639 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:44.405138 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18133bde-5626-4516-b126-07b620c0d639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:44.459764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-18133bde-5626-4516-b126-07b620c0d639 None None] MFA Rules not processed for user `70b57718e8fa417386db06fd42a0fde2`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:44.460929 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-18133bde-5626-4516-b126-07b620c0d639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:44.578876 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1217/2446] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:00:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:44.593022 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c677f2-ae2e-42ce-a16d-0e609c37ca60 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:44.596664 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-31c677f2-ae2e-42ce-a16d-0e609c37ca60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:44.695163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c677f2-ae2e-42ce-a16d-0e609c37ca60 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:44.699151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e85569-ca56-4c81-b65e-baf786268dcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:44.699825 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e85569-ca56-4c81-b65e-baf786268dcd service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:44.700081 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e85569-ca56-4c81-b65e-baf786268dcd service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:44.700354 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e85569-ca56-4c81-b65e-baf786268dcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:44.712047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e85569-ca56-4c81-b65e-baf786268dcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:44.713529 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e85569-ca56-4c81-b65e-baf786268dcd service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:44.718114 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:44.726401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1230/2447] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:44 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:44.959303 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18133bde-5626-4516-b126-07b620c0d639 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:44.970356 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18133bde-5626-4516-b126-07b620c0d639 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:44.974430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b272c2e-be10-47ad-ad6d-dfceac85ee03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:44.975100 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b272c2e-be10-47ad-ad6d-dfceac85ee03 service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:44.975380 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b272c2e-be10-47ad-ad6d-dfceac85ee03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:44.975650 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b272c2e-be10-47ad-ad6d-dfceac85ee03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:45.040742 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:45.059334 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b272c2e-be10-47ad-ad6d-dfceac85ee03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:45.063052 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b272c2e-be10-47ad-ad6d-dfceac85ee03 service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:45.081774 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1218/2448] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:00:54.935493 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:55.129989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=6 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:57.314453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b2d45-097d-4867-bfe2-873343a8691e None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:57.324896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b2d45-097d-4867-bfe2-873343a8691e None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:57.324896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b2d45-097d-4867-bfe2-873343a8691e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:57.369617 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-663b2d45-097d-4867-bfe2-873343a8691e None None] MFA Rules not processed for user `9646662b0b3048f79ad5d2b9602f10a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:57.373188 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-663b2d45-097d-4867-bfe2-873343a8691e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:57.463208 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1231/2449] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Aug 8 17:00:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:57.673690 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b272c2e-be10-47ad-ad6d-dfceac85ee03 service cinder] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:57.675371 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-0b272c2e-be10-47ad-ad6d-dfceac85ee03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:57.781848 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b272c2e-be10-47ad-ad6d-dfceac85ee03 service cinder] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:57.787366 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae87359f-d0db-4aef-b65a-e3196497365a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c606decb832444e38a194b98f8d15f8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:57.788057 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87359f-d0db-4aef-b65a-e3196497365a service swift] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:57.788406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87359f-d0db-4aef-b65a-e3196497365a service swift] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:57.788714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87359f-d0db-4aef-b65a-e3196497365a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:57.798476 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae87359f-d0db-4aef-b65a-e3196497365a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:57.800379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae87359f-d0db-4aef-b65a-e3196497365a service swift] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:57.810039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1219/2450] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Aug 8 17:00:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 17:00:57.845006 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b2d45-097d-4867-bfe2-873343a8691e None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:57.969932 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b2d45-097d-4867-bfe2-873343a8691e None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:57.979893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf27e4-6339-49d0-9ee6-b593d2b46c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:57.981004 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf27e4-6339-49d0-9ee6-b593d2b46c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:57.981412 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf27e4-6339-49d0-9ee6-b593d2b46c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:57.981787 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf27e4-6339-49d0-9ee6-b593d2b46c55 admin admin] PATH_INFO: `/v3/users/0270311c3b6c4a2b8534fcaefc1c95c6` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:57.983688 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf27e4-6339-49d0-9ee6-b593d2b46c55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0270311c3b6c4a2b8534fcaefc1c95c6)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:57.987159 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf27e4-6339-49d0-9ee6-b593d2b46c55 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:58.255952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-7ddf27e4-6339-49d0-9ee6-b593d2b46c55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0270311c3b6c4a2b8534fcaefc1c95c6'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:58.342744 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-7ddf27e4-6339-49d0-9ee6-b593d2b46c55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0270311c3b6c4a2b8534fcaefc1c95c6'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:58.344808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-7ddf27e4-6339-49d0-9ee6-b593d2b46c55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0270311c3b6c4a2b8534fcaefc1c95c6'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:58.356810 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-7ddf27e4-6339-49d0-9ee6-b593d2b46c55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0270311c3b6c4a2b8534fcaefc1c95c6'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:58.374714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-7ddf27e4-6339-49d0-9ee6-b593d2b46c55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0270311c3b6c4a2b8534fcaefc1c95c6'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:58.387879 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1232/2451] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:57 2022] DELETE /identity/v3/users/0270311c3b6c4a2b8534fcaefc1c95c6 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:58.390131 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ce669-5d01-44c3-8e40-0a71b9cf1cee None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:58.390919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ce669-5d01-44c3-8e40-0a71b9cf1cee None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:58.391052 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ce669-5d01-44c3-8e40-0a71b9cf1cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:58.425065 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-273ce669-5d01-44c3-8e40-0a71b9cf1cee None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:58.433639 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-273ce669-5d01-44c3-8e40-0a71b9cf1cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:58.559137 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1220/2452] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:00:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:58.574738 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf27e4-6339-49d0-9ee6-b593d2b46c55 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:58.576426 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-7ddf27e4-6339-49d0-9ee6-b593d2b46c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:58.597329 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:58.744746 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf27e4-6339-49d0-9ee6-b593d2b46c55 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:58.754332 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076fec95-dd74-422d-a37f-b9146e6b2b39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:58.755867 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076fec95-dd74-422d-a37f-b9146e6b2b39 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:58.756516 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076fec95-dd74-422d-a37f-b9146e6b2b39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:58.757232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076fec95-dd74-422d-a37f-b9146e6b2b39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:58.779028 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076fec95-dd74-422d-a37f-b9146e6b2b39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:58.781502 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076fec95-dd74-422d-a37f-b9146e6b2b39 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:58.795339 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1233/2453] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:59.457160 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ce669-5d01-44c3-8e40-0a71b9cf1cee None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:59.460587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-273ce669-5d01-44c3-8e40-0a71b9cf1cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:59.502967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ce669-5d01-44c3-8e40-0a71b9cf1cee None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:59.520927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164fa57-a86b-4d6f-a282-86d6fd6c2b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:59.521592 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164fa57-a86b-4d6f-a282-86d6fd6c2b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:59.521867 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164fa57-a86b-4d6f-a282-86d6fd6c2b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:59.522526 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164fa57-a86b-4d6f-a282-86d6fd6c2b4e admin admin] PATH_INFO: `/v3/projects/3323ecb089054f5491b4f93669194b7a` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:59.529004 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c164fa57-a86b-4d6f-a282-86d6fd6c2b4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3323ecb089054f5491b4f93669194b7a)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:59.529004 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c164fa57-a86b-4d6f-a282-86d6fd6c2b4e admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:59.675207 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c164fa57-a86b-4d6f-a282-86d6fd6c2b4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3323ecb089054f5491b4f93669194b7a', 'request_id': 'req-c164fa57-a86b-4d6f-a282-86d6fd6c2b4e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:59.691399 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c164fa57-a86b-4d6f-a282-86d6fd6c2b4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3323ecb089054f5491b4f93669194b7a', 'request_id': 'req-c164fa57-a86b-4d6f-a282-86d6fd6c2b4e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:59.709471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c164fa57-a86b-4d6f-a282-86d6fd6c2b4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3323ecb089054f5491b4f93669194b7a', 'request_id': 'req-c164fa57-a86b-4d6f-a282-86d6fd6c2b4e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:59.835045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c164fa57-a86b-4d6f-a282-86d6fd6c2b4e admin admin] The token cache is being invalidate because project 3323ecb089054f5491b4f93669194b7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:59.835794 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c164fa57-a86b-4d6f-a282-86d6fd6c2b4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:59.837143 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c164fa57-a86b-4d6f-a282-86d6fd6c2b4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:59.840517 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1221/2454] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:59 2022] DELETE /identity/v3/projects/3323ecb089054f5491b4f93669194b7a => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:59.847763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076fec95-dd74-422d-a37f-b9146e6b2b39 service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:59.892854 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076fec95-dd74-422d-a37f-b9146e6b2b39 service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:00.060447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b149b-447e-4c89-806a-ea003cf70a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:00.060447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b149b-447e-4c89-806a-ea003cf70a14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:00.060447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b149b-447e-4c89-806a-ea003cf70a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:00.060447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b149b-447e-4c89-806a-ea003cf70a14 admin admin] PATH_INFO: `/v3/users/70b57718e8fa417386db06fd42a0fde2` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:00.060894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59b149b-447e-4c89-806a-ea003cf70a14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70b57718e8fa417386db06fd42a0fde2)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:00.060894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59b149b-447e-4c89-806a-ea003cf70a14 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:00.328664 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-f59b149b-447e-4c89-806a-ea003cf70a14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70b57718e8fa417386db06fd42a0fde2'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:00.462309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-f59b149b-447e-4c89-806a-ea003cf70a14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70b57718e8fa417386db06fd42a0fde2'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:00.463949 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-f59b149b-447e-4c89-806a-ea003cf70a14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70b57718e8fa417386db06fd42a0fde2'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:00.489284 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-f59b149b-447e-4c89-806a-ea003cf70a14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70b57718e8fa417386db06fd42a0fde2'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:00.531060 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-f59b149b-447e-4c89-806a-ea003cf70a14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70b57718e8fa417386db06fd42a0fde2'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:00.539693 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1234/2455] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:59 2022] DELETE /identity/v3/users/70b57718e8fa417386db06fd42a0fde2 => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:00.821680 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164fa57-a86b-4d6f-a282-86d6fd6c2b4e admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:00.823548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-c164fa57-a86b-4d6f-a282-86d6fd6c2b4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:00.856350 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164fa57-a86b-4d6f-a282-86d6fd6c2b4e admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:00.878764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:01.060803 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081f3dc-3fd6-4be3-9c2c-f28a7f0d5880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:01.062071 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081f3dc-3fd6-4be3-9c2c-f28a7f0d5880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:01.062377 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081f3dc-3fd6-4be3-9c2c-f28a7f0d5880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:01.063352 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081f3dc-3fd6-4be3-9c2c-f28a7f0d5880 admin admin] PATH_INFO: `/v3/users/a7e1b4d982e348428e97d8d48946f2be` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:01.068079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b081f3dc-3fd6-4be3-9c2c-f28a7f0d5880 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7e1b4d982e348428e97d8d48946f2be)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:01.070147 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b081f3dc-3fd6-4be3-9c2c-f28a7f0d5880 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:01.347075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-b081f3dc-3fd6-4be3-9c2c-f28a7f0d5880 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7e1b4d982e348428e97d8d48946f2be'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:01.417796 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b149b-447e-4c89-806a-ea003cf70a14 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:01.428366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f59b149b-447e-4c89-806a-ea003cf70a14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:01.444607 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-b081f3dc-3fd6-4be3-9c2c-f28a7f0d5880 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7e1b4d982e348428e97d8d48946f2be'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:01.447544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-b081f3dc-3fd6-4be3-9c2c-f28a7f0d5880 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7e1b4d982e348428e97d8d48946f2be'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:01.459370 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b149b-447e-4c89-806a-ea003cf70a14 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:01.466875 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-b081f3dc-3fd6-4be3-9c2c-f28a7f0d5880 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7e1b4d982e348428e97d8d48946f2be'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:01.469714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-b081f3dc-3fd6-4be3-9c2c-f28a7f0d5880 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7e1b4d982e348428e97d8d48946f2be'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:01.470217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ea2a4-4f71-42b7-a7cb-94eb88981d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:01.471353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ea2a4-4f71-42b7-a7cb-94eb88981d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:01.472096 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ea2a4-4f71-42b7-a7cb-94eb88981d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:01.472662 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ea2a4-4f71-42b7-a7cb-94eb88981d48 admin admin] PATH_INFO: `/v3/projects/ee8ab01efcef49d7a9df6f67af2638fe` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:01.476021 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ea2a4-4f71-42b7-a7cb-94eb88981d48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee8ab01efcef49d7a9df6f67af2638fe)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:01.479214 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ea2a4-4f71-42b7-a7cb-94eb88981d48 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:01.509340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1222/2456] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:00 2022] DELETE /identity/v3/users/a7e1b4d982e348428e97d8d48946f2be => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:01.515787 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9c876-eb3b-48cb-b5dc-ce56a2cfdcdb None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:01.516097 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9c876-eb3b-48cb-b5dc-ce56a2cfdcdb None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:01.516393 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9c876-eb3b-48cb-b5dc-ce56a2cfdcdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:01.542238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-01f9c876-eb3b-48cb-b5dc-ce56a2cfdcdb None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:01.543572 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-01f9c876-eb3b-48cb-b5dc-ce56a2cfdcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:01.895310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1223/2457] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:01.904022 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9c876-eb3b-48cb-b5dc-ce56a2cfdcdb None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:01.906471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-01f9c876-eb3b-48cb-b5dc-ce56a2cfdcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:01.953805 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:01.980466 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9c876-eb3b-48cb-b5dc-ce56a2cfdcdb None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:01.984787 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2b523-b5e7-40c7-891f-3a40f4e40158 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:01.985414 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2b523-b5e7-40c7-891f-3a40f4e40158 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:01.985689 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2b523-b5e7-40c7-891f-3a40f4e40158 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:01.996608 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2b523-b5e7-40c7-891f-3a40f4e40158 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:02.001527 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2b523-b5e7-40c7-891f-3a40f4e40158 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:02.003061 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2b523-b5e7-40c7-891f-3a40f4e40158 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:02.013912 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1224/2458] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:02.129123 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:02.161192 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-536ea2a4-4f71-42b7-a7cb-94eb88981d48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee8ab01efcef49d7a9df6f67af2638fe', 'request_id': 'req-536ea2a4-4f71-42b7-a7cb-94eb88981d48'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:02.288443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-536ea2a4-4f71-42b7-a7cb-94eb88981d48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee8ab01efcef49d7a9df6f67af2638fe', 'request_id': 'req-536ea2a4-4f71-42b7-a7cb-94eb88981d48'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:02.299543 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-536ea2a4-4f71-42b7-a7cb-94eb88981d48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee8ab01efcef49d7a9df6f67af2638fe', 'request_id': 'req-536ea2a4-4f71-42b7-a7cb-94eb88981d48'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:02.366741 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-536ea2a4-4f71-42b7-a7cb-94eb88981d48 admin admin] The token cache is being invalidate because project ee8ab01efcef49d7a9df6f67af2638fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:02.367651 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-536ea2a4-4f71-42b7-a7cb-94eb88981d48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:02.368588 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-536ea2a4-4f71-42b7-a7cb-94eb88981d48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:02.372733 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1235/2459] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:01 2022] DELETE /identity/v3/projects/ee8ab01efcef49d7a9df6f67af2638fe => generated 0 bytes in 955 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:02.378557 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a82509-c4fb-4576-9525-f64a642c450d None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:02.379092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a82509-c4fb-4576-9525-f64a642c450d None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:02.379379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a82509-c4fb-4576-9525-f64a642c450d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:02.414352 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-f2a82509-c4fb-4576-9525-f64a642c450d None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:02.416751 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-f2a82509-c4fb-4576-9525-f64a642c450d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:02.633825 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1225/2460] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:02.649513 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadc218-adb0-4498-8792-09d1363b1add None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:02.650088 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadc218-adb0-4498-8792-09d1363b1add None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:02.650405 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadc218-adb0-4498-8792-09d1363b1add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:02.689877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-6dadc218-adb0-4498-8792-09d1363b1add None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:02.692282 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-6dadc218-adb0-4498-8792-09d1363b1add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:02.720416 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a82509-c4fb-4576-9525-f64a642c450d None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:02.721552 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1236/2461] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:02.745479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a82509-c4fb-4576-9525-f64a642c450d None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:02.767554 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da6ac0-7923-4206-b9e6-94abbe7bc73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:02.768917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da6ac0-7923-4206-b9e6-94abbe7bc73b admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:02.769119 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da6ac0-7923-4206-b9e6-94abbe7bc73b admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:02.769539 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da6ac0-7923-4206-b9e6-94abbe7bc73b admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:02.770418 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da6ac0-7923-4206-b9e6-94abbe7bc73b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:02.770939 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da6ac0-7923-4206-b9e6-94abbe7bc73b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:02.773104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da6ac0-7923-4206-b9e6-94abbe7bc73b admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:02.813961 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1226/2462] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:02.815422 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dadc218-adb0-4498-8792-09d1363b1add None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:02.831145 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da6ac0-7923-4206-b9e6-94abbe7bc73b admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:02.834555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dadc218-adb0-4498-8792-09d1363b1add None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:02.840338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-b3da6ac0-7923-4206-b9e6-94abbe7bc73b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:02.842996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07908ec-036c-4a77-a3cd-183d012ae7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:02.844932 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07908ec-036c-4a77-a3cd-183d012ae7da admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:02.845406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07908ec-036c-4a77-a3cd-183d012ae7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:02.846007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07908ec-036c-4a77-a3cd-183d012ae7da admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:02.846731 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07908ec-036c-4a77-a3cd-183d012ae7da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:02.851204 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07908ec-036c-4a77-a3cd-183d012ae7da admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:02.866767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da6ac0-7923-4206-b9e6-94abbe7bc73b admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:02.887758 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbe4b9-0360-4efb-b03c-6ee4a3861fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:02.891738 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbe4b9-0360-4efb-b03c-6ee4a3861fe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:02.891738 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbe4b9-0360-4efb-b03c-6ee4a3861fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:02.891738 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbe4b9-0360-4efb-b03c-6ee4a3861fe4 admin admin] PATH_INFO: `/v3/projects/6edade7c5d3e42ba97a28e0e2bf52d26` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:02.891738 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcbe4b9-0360-4efb-b03c-6ee4a3861fe4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6edade7c5d3e42ba97a28e0e2bf52d26)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:02.892837 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcbe4b9-0360-4efb-b03c-6ee4a3861fe4 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:02.993457 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1237/2463] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:02 2022] POST /identity/v3/projects => generated 381 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:02.995537 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07908ec-036c-4a77-a3cd-183d012ae7da admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:03.083461 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7fcbe4b9-0360-4efb-b03c-6ee4a3861fe4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6edade7c5d3e42ba97a28e0e2bf52d26', 'request_id': 'req-7fcbe4b9-0360-4efb-b03c-6ee4a3861fe4'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:03.090713 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7fcbe4b9-0360-4efb-b03c-6ee4a3861fe4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6edade7c5d3e42ba97a28e0e2bf52d26', 'request_id': 'req-7fcbe4b9-0360-4efb-b03c-6ee4a3861fe4'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:03.099633 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7fcbe4b9-0360-4efb-b03c-6ee4a3861fe4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6edade7c5d3e42ba97a28e0e2bf52d26', 'request_id': 'req-7fcbe4b9-0360-4efb-b03c-6ee4a3861fe4'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:03.137105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07908ec-036c-4a77-a3cd-183d012ae7da admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:03.177877 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7fcbe4b9-0360-4efb-b03c-6ee4a3861fe4 admin admin] The token cache is being invalidate because project 6edade7c5d3e42ba97a28e0e2bf52d26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:03.178382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7fcbe4b9-0360-4efb-b03c-6ee4a3861fe4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:03.179562 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7fcbe4b9-0360-4efb-b03c-6ee4a3861fe4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:03.182763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1227/2464] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:02 2022] DELETE /identity/v3/projects/6edade7c5d3e42ba97a28e0e2bf52d26 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:03.204869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe65c2-f7dc-4a54-b887-543bf4c43fae None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:03.204869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe65c2-f7dc-4a54-b887-543bf4c43fae None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:03.204869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe65c2-f7dc-4a54-b887-543bf4c43fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:03.222836 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-9dbe65c2-f7dc-4a54-b887-543bf4c43fae None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:03.225374 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-9dbe65c2-f7dc-4a54-b887-543bf4c43fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:03.256754 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3d073-51b0-4437-8337-d09dd52a2f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:03.257608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3d073-51b0-4437-8337-d09dd52a2f4b admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:03.258029 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3d073-51b0-4437-8337-d09dd52a2f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:03.258429 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3d073-51b0-4437-8337-d09dd52a2f4b admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:03.259521 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3d073-51b0-4437-8337-d09dd52a2f4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:03.270358 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3d073-51b0-4437-8337-d09dd52a2f4b admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:03.329983 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:03.368964 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1228/2465] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:03.383085 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc00cc-8b0f-4b5a-9656-e4b34253ea18 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:03.384002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc00cc-8b0f-4b5a-9656-e4b34253ea18 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:03.384754 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc00cc-8b0f-4b5a-9656-e4b34253ea18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:03.409208 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1238/2466] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:02 2022] POST /identity/v3/users => generated 291 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:03.415243 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3d073-51b0-4437-8337-d09dd52a2f4b admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:03.422083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-02b3d073-51b0-4437-8337-d09dd52a2f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:03.431728 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-47bc00cc-8b0f-4b5a-9656-e4b34253ea18 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:03.434395 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-47bc00cc-8b0f-4b5a-9656-e4b34253ea18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:03.441096 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3d073-51b0-4437-8337-d09dd52a2f4b admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:03.448439 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1229/2467] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:03.454735 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc00cc-8b0f-4b5a-9656-e4b34253ea18 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:03.461493 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c253d82-cfc9-4355-9d1c-cceaf59ea7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:03.463614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c253d82-cfc9-4355-9d1c-cceaf59ea7e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:03.464151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c253d82-cfc9-4355-9d1c-cceaf59ea7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:03.464923 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c253d82-cfc9-4355-9d1c-cceaf59ea7e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:03.465825 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c253d82-cfc9-4355-9d1c-cceaf59ea7e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:03.468781 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c253d82-cfc9-4355-9d1c-cceaf59ea7e4 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:03.486068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc00cc-8b0f-4b5a-9656-e4b34253ea18 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:03.490275 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1239/2468] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:03.504425 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c253d82-cfc9-4355-9d1c-cceaf59ea7e4 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:03.505484 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7e499-94ce-4eee-822d-f8b6bc9cba52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:03.507805 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7e499-94ce-4eee-822d-f8b6bc9cba52 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:03.508325 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7e499-94ce-4eee-822d-f8b6bc9cba52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:03.508674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7e499-94ce-4eee-822d-f8b6bc9cba52 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:03.509435 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7e499-94ce-4eee-822d-f8b6bc9cba52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:03.509807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7e499-94ce-4eee-822d-f8b6bc9cba52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:03.511674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7e499-94ce-4eee-822d-f8b6bc9cba52 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:03.543734 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c253d82-cfc9-4355-9d1c-cceaf59ea7e4 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:03.544150 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1230/2469] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:03.549659 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7e499-94ce-4eee-822d-f8b6bc9cba52 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:03.565417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19900a-5caa-485f-b2ba-0111837e1365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:03.566055 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19900a-5caa-485f-b2ba-0111837e1365 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:03.566333 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19900a-5caa-485f-b2ba-0111837e1365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:03.566641 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19900a-5caa-485f-b2ba-0111837e1365 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:03.567341 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd19900a-5caa-485f-b2ba-0111837e1365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:03.569061 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd19900a-5caa-485f-b2ba-0111837e1365 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:03.578144 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7e499-94ce-4eee-822d-f8b6bc9cba52 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:03.582067 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:03.595559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30355158-c43e-4147-9fdb-5a62409b258f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:03.596185 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30355158-c43e-4147-9fdb-5a62409b258f admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:03.596504 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30355158-c43e-4147-9fdb-5a62409b258f admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:03.596782 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30355158-c43e-4147-9fdb-5a62409b258f admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:03.597433 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30355158-c43e-4147-9fdb-5a62409b258f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:03.599511 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30355158-c43e-4147-9fdb-5a62409b258f admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:03.602233 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1240/2470] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:03.608897 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19900a-5caa-485f-b2ba-0111837e1365 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:03.624578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19900a-5caa-485f-b2ba-0111837e1365 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:03.637457 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10242236-c377-43e6-9cb9-f77e3e1b8936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:03.639076 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10242236-c377-43e6-9cb9-f77e3e1b8936 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:03.640882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10242236-c377-43e6-9cb9-f77e3e1b8936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:03.641823 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10242236-c377-43e6-9cb9-f77e3e1b8936 admin admin] PATH_INFO: `/v3/projects/ee4a78ad41b3453fbf056f453dbb2a00/users/8a8be8f0214f49d2868b9223c6168209/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:03.674542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10242236-c377-43e6-9cb9-f77e3e1b8936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee4a78ad41b3453fbf056f453dbb2a00, user_id=8a8be8f0214f49d2868b9223c6168209, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:03.677802 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10242236-c377-43e6-9cb9-f77e3e1b8936 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:03.731941 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1231/2471] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:03 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:03.734624 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30355158-c43e-4147-9fdb-5a62409b258f admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:03.846609 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1241/2472] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:03 2022] PUT /identity/v3/projects/ee4a78ad41b3453fbf056f453dbb2a00/users/8a8be8f0214f49d2868b9223c6168209/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:03.848501 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c6fb9-f099-499d-9bb4-999ab873c42b None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:03.851678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c6fb9-f099-499d-9bb4-999ab873c42b None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:03.852304 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c6fb9-f099-499d-9bb4-999ab873c42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:03.871145 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30355158-c43e-4147-9fdb-5a62409b258f admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:03.952030 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-5e7c6fb9-f099-499d-9bb4-999ab873c42b None None] MFA Rules not processed for user `8a8be8f0214f49d2868b9223c6168209`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:03.954478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-5e7c6fb9-f099-499d-9bb4-999ab873c42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:04.087153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73875b37-c550-4379-af79-01bb6be430ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:04.088033 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73875b37-c550-4379-af79-01bb6be430ee admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:04.088033 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73875b37-c550-4379-af79-01bb6be430ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:04.088344 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73875b37-c550-4379-af79-01bb6be430ee admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:04.088930 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73875b37-c550-4379-af79-01bb6be430ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:04.092004 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73875b37-c550-4379-af79-01bb6be430ee admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:04.127045 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1242/2473] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:04.128055 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c6fb9-f099-499d-9bb4-999ab873c42b None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:04.145269 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c6fb9-f099-499d-9bb4-999ab873c42b None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:04.152545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b73dd8-d6c6-47f7-9b07-87edc8dd2f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:04.153177 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b73dd8-d6c6-47f7-9b07-87edc8dd2f30 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:04.153453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b73dd8-d6c6-47f7-9b07-87edc8dd2f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:04.153722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b73dd8-d6c6-47f7-9b07-87edc8dd2f30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:04.154356 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b73dd8-d6c6-47f7-9b07-87edc8dd2f30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:04.158168 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b73dd8-d6c6-47f7-9b07-87edc8dd2f30 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:04.242669 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1232/2474] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:03 2022] POST /identity/v3/users => generated 293 bytes in 505 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:04.244285 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73875b37-c550-4379-af79-01bb6be430ee admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:04.274066 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1243/2475] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:04 2022] POST /identity/v3/projects => generated 381 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:04.278880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b73dd8-d6c6-47f7-9b07-87edc8dd2f30 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:04.288352 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73875b37-c550-4379-af79-01bb6be430ee admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:04.396196 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:04.424282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56323a-8b6a-4a19-aeec-06878f115040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:04.425076 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56323a-8b6a-4a19-aeec-06878f115040 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:04.425354 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56323a-8b6a-4a19-aeec-06878f115040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:04.425638 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56323a-8b6a-4a19-aeec-06878f115040 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:04.426736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56323a-8b6a-4a19-aeec-06878f115040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:04.429499 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56323a-8b6a-4a19-aeec-06878f115040 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:04.444837 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1233/2476] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:04.450087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56323a-8b6a-4a19-aeec-06878f115040 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:04.460834 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b73dd8-d6c6-47f7-9b07-87edc8dd2f30 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:04.465809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56323a-8b6a-4a19-aeec-06878f115040 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:04.470657 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc4453d-540c-4d33-a559-0c1c9ff811f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:04.471968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4453d-540c-4d33-a559-0c1c9ff811f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:04.472592 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4453d-540c-4d33-a559-0c1c9ff811f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:04.473210 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4453d-540c-4d33-a559-0c1c9ff811f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:04.473816 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95221e-0df0-42a3-bab8-a08cf92b4e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:04.474440 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95221e-0df0-42a3-bab8-a08cf92b4e8a admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:04.474593 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc4453d-540c-4d33-a559-0c1c9ff811f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:04.474771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95221e-0df0-42a3-bab8-a08cf92b4e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:04.475037 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95221e-0df0-42a3-bab8-a08cf92b4e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:04.476255 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95221e-0df0-42a3-bab8-a08cf92b4e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:04.479021 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc4453d-540c-4d33-a559-0c1c9ff811f6 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:04.479180 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95221e-0df0-42a3-bab8-a08cf92b4e8a admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:04.498771 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1234/2477] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:04.502114 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95221e-0df0-42a3-bab8-a08cf92b4e8a admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:04.524330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95221e-0df0-42a3-bab8-a08cf92b4e8a admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:04.535882 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4060d6-84ab-41a1-9348-4e4ae3526e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:04.536973 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4060d6-84ab-41a1-9348-4e4ae3526e4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:04.537244 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4060d6-84ab-41a1-9348-4e4ae3526e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:04.537762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4060d6-84ab-41a1-9348-4e4ae3526e4c admin admin] PATH_INFO: `/v3/projects/df87dc04301c41bca9a33febf716f62d/users/a699c03a53e84c5ba311d0700c8e0fc7/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:04.571541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4060d6-84ab-41a1-9348-4e4ae3526e4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=df87dc04301c41bca9a33febf716f62d, user_id=a699c03a53e84c5ba311d0700c8e0fc7, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:04.578191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4060d6-84ab-41a1-9348-4e4ae3526e4c admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:04.645727 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1244/2478] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:04 2022] POST /identity/v3/users => generated 297 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:04.650313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc4453d-540c-4d33-a559-0c1c9ff811f6 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:04.691865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1235/2479] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:04 2022] PUT /identity/v3/projects/df87dc04301c41bca9a33febf716f62d/users/a699c03a53e84c5ba311d0700c8e0fc7/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:04.695236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc4453d-540c-4d33-a559-0c1c9ff811f6 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:04.714173 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363d6d3-5106-49cb-bb85-991056794cd6 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:04.714608 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363d6d3-5106-49cb-bb85-991056794cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:04.715110 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363d6d3-5106-49cb-bb85-991056794cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:04.750637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:04.785484 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-9363d6d3-5106-49cb-bb85-991056794cd6 None None] MFA Rules not processed for user `a699c03a53e84c5ba311d0700c8e0fc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:04.788386 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-9363d6d3-5106-49cb-bb85-991056794cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:04.810462 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fae9fe-3a52-4b22-8958-9e0f18788e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:04.812493 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fae9fe-3a52-4b22-8958-9e0f18788e77 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:04.812727 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fae9fe-3a52-4b22-8958-9e0f18788e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:04.813166 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fae9fe-3a52-4b22-8958-9e0f18788e77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:04.814501 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fae9fe-3a52-4b22-8958-9e0f18788e77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:04.817755 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fae9fe-3a52-4b22-8958-9e0f18788e77 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:04.858864 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1245/2480] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:04.864261 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fae9fe-3a52-4b22-8958-9e0f18788e77 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:04.887671 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fae9fe-3a52-4b22-8958-9e0f18788e77 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:04.901735 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61bc43-a14d-4374-b05f-b3a669fa5f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:04.902872 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61bc43-a14d-4374-b05f-b3a669fa5f9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:04.903157 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61bc43-a14d-4374-b05f-b3a669fa5f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:04.903431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61bc43-a14d-4374-b05f-b3a669fa5f9d admin admin] PATH_INFO: `/v3/projects/13ab433331e14202aa18a558dd0749b5/users/124462a12cd64ad1abddac93c2b52b7d/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:04.934997 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e61bc43-a14d-4374-b05f-b3a669fa5f9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=13ab433331e14202aa18a558dd0749b5, user_id=124462a12cd64ad1abddac93c2b52b7d, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:04.940734 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e61bc43-a14d-4374-b05f-b3a669fa5f9d admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:04.951219 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1236/2481] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:04.971041 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363d6d3-5106-49cb-bb85-991056794cd6 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:04.975505 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-9363d6d3-5106-49cb-bb85-991056794cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:05.095525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1246/2482] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:04 2022] PUT /identity/v3/projects/13ab433331e14202aa18a558dd0749b5/users/124462a12cd64ad1abddac93c2b52b7d/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:05.104268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61bc43-a14d-4374-b05f-b3a669fa5f9d admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:05.107712 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363d6d3-5106-49cb-bb85-991056794cd6 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:05.172991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:05.179006 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790d624-b1b4-473f-af1a-e2f35e3401b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:05.179805 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790d624-b1b4-473f-af1a-e2f35e3401b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:05.180176 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790d624-b1b4-473f-af1a-e2f35e3401b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:05.180487 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790d624-b1b4-473f-af1a-e2f35e3401b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:05.291435 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61bc43-a14d-4374-b05f-b3a669fa5f9d admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:05.297902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d464410-9f5a-42af-971d-7ce678574ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:05.298548 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d464410-9f5a-42af-971d-7ce678574ceb admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:05.298882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d464410-9f5a-42af-971d-7ce678574ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:05.299157 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d464410-9f5a-42af-971d-7ce678574ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:05.305574 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d464410-9f5a-42af-971d-7ce678574ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:05.310139 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d464410-9f5a-42af-971d-7ce678574ceb admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:05.321994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1247/2483] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:05.326555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d464410-9f5a-42af-971d-7ce678574ceb admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:05.340484 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790d624-b1b4-473f-af1a-e2f35e3401b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:05.343485 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790d624-b1b4-473f-af1a-e2f35e3401b6 service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:05.351962 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d464410-9f5a-42af-971d-7ce678574ceb admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:05.362856 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff590129-6810-49dc-822c-337a0faa918b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:05.364801 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff590129-6810-49dc-822c-337a0faa918b admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:05.365234 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff590129-6810-49dc-822c-337a0faa918b admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:05.365514 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff590129-6810-49dc-822c-337a0faa918b admin admin] PATH_INFO: `/v3/projects/13ab433331e14202aa18a558dd0749b5/users/124462a12cd64ad1abddac93c2b52b7d/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:05.366867 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1237/2484] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:05.382323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff590129-6810-49dc-822c-337a0faa918b admin admin] RBAC: Authorizing `identity:create_grant(project_id=13ab433331e14202aa18a558dd0749b5, user_id=124462a12cd64ad1abddac93c2b52b7d, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:05.385834 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff590129-6810-49dc-822c-337a0faa918b admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:05.415981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1248/2485] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:05 2022] PUT /identity/v3/projects/13ab433331e14202aa18a558dd0749b5/users/124462a12cd64ad1abddac93c2b52b7d/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:05.443686 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446fd25-8298-4eb5-afa5-172e6f1a8de7 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:05.445708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446fd25-8298-4eb5-afa5-172e6f1a8de7 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:05.445897 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446fd25-8298-4eb5-afa5-172e6f1a8de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:05.511035 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-3446fd25-8298-4eb5-afa5-172e6f1a8de7 None None] MFA Rules not processed for user `124462a12cd64ad1abddac93c2b52b7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:05.513099 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-3446fd25-8298-4eb5-afa5-172e6f1a8de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:05.678944 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1238/2486] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:05.701164 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff590129-6810-49dc-822c-337a0faa918b admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:05.704485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ff590129-6810-49dc-822c-337a0faa918b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:05.830001 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff590129-6810-49dc-822c-337a0faa918b admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:05.833503 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a4326-9fde-492b-b0dd-330a1acc7de3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:05.834140 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a4326-9fde-492b-b0dd-330a1acc7de3 service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:05.834424 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a4326-9fde-492b-b0dd-330a1acc7de3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:05.834724 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a4326-9fde-492b-b0dd-330a1acc7de3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:05.847416 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a4326-9fde-492b-b0dd-330a1acc7de3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:05.848800 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a4326-9fde-492b-b0dd-330a1acc7de3 service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:05.862610 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1249/2487] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:06.035367 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446fd25-8298-4eb5-afa5-172e6f1a8de7 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:06.047598 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446fd25-8298-4eb5-afa5-172e6f1a8de7 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:06.052918 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ec1d6-18fb-445c-b18a-12133b1a1d02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:06.052918 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ec1d6-18fb-445c-b18a-12133b1a1d02 service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:06.052918 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ec1d6-18fb-445c-b18a-12133b1a1d02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:06.053475 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ec1d6-18fb-445c-b18a-12133b1a1d02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:06.127282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ec1d6-18fb-445c-b18a-12133b1a1d02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:06.128790 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ec1d6-18fb-445c-b18a-12133b1a1d02 service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:06.141537 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1239/2488] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:06 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:13.704010 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a4326-9fde-492b-b0dd-330a1acc7de3 service cinder] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:13.758046 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:13.808259 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a4326-9fde-492b-b0dd-330a1acc7de3 service cinder] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:13.816761 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52af3dd-6462-49b9-a7ec-c9cec5bf870a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:13.817579 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52af3dd-6462-49b9-a7ec-c9cec5bf870a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:13.818067 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52af3dd-6462-49b9-a7ec-c9cec5bf870a admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:13.818450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52af3dd-6462-49b9-a7ec-c9cec5bf870a admin admin] PATH_INFO: `/v3/users/8a8be8f0214f49d2868b9223c6168209` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:13.820898 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52af3dd-6462-49b9-a7ec-c9cec5bf870a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a8be8f0214f49d2868b9223c6168209)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:13.823142 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52af3dd-6462-49b9-a7ec-c9cec5bf870a admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:14.071686 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b52af3dd-6462-49b9-a7ec-c9cec5bf870a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a8be8f0214f49d2868b9223c6168209'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:14.141328 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b52af3dd-6462-49b9-a7ec-c9cec5bf870a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a8be8f0214f49d2868b9223c6168209'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:14.143709 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b52af3dd-6462-49b9-a7ec-c9cec5bf870a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a8be8f0214f49d2868b9223c6168209'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:14.163445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b52af3dd-6462-49b9-a7ec-c9cec5bf870a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a8be8f0214f49d2868b9223c6168209'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:14.182618 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b52af3dd-6462-49b9-a7ec-c9cec5bf870a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a8be8f0214f49d2868b9223c6168209'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:14.204907 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1250/2489] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:13 2022] DELETE /identity/v3/users/8a8be8f0214f49d2868b9223c6168209 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:14.209518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0f8dd-7de6-478e-8458-b496bf6d3b5b None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:14.210809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0f8dd-7de6-478e-8458-b496bf6d3b5b None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:14.211152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0f8dd-7de6-478e-8458-b496bf6d3b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:14.242143 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-22a0f8dd-7de6-478e-8458-b496bf6d3b5b None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:14.244821 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-22a0f8dd-7de6-478e-8458-b496bf6d3b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:14.289578 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:14.352707 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1240/2490] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:14.362070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52af3dd-6462-49b9-a7ec-c9cec5bf870a admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:14.363637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b52af3dd-6462-49b9-a7ec-c9cec5bf870a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:14.456869 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52af3dd-6462-49b9-a7ec-c9cec5bf870a admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:14.462449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75201b-50bc-4a83-85d3-cd1596aad384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:14.467778 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75201b-50bc-4a83-85d3-cd1596aad384 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:14.467778 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75201b-50bc-4a83-85d3-cd1596aad384 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:14.467778 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75201b-50bc-4a83-85d3-cd1596aad384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:14.476886 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75201b-50bc-4a83-85d3-cd1596aad384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:14.478422 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75201b-50bc-4a83-85d3-cd1596aad384 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:14.493940 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1251/2491] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:15.046163 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0f8dd-7de6-478e-8458-b496bf6d3b5b None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:15.048226 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-22a0f8dd-7de6-478e-8458-b496bf6d3b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:15.065835 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0f8dd-7de6-478e-8458-b496bf6d3b5b None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:15.073748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252c9cc-b6d8-4a8c-9dbc-ae46b7590f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:15.074888 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252c9cc-b6d8-4a8c-9dbc-ae46b7590f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:15.075387 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252c9cc-b6d8-4a8c-9dbc-ae46b7590f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:15.075875 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252c9cc-b6d8-4a8c-9dbc-ae46b7590f20 admin admin] PATH_INFO: `/v3/projects/ee4a78ad41b3453fbf056f453dbb2a00` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:15.077869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252c9cc-b6d8-4a8c-9dbc-ae46b7590f20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee4a78ad41b3453fbf056f453dbb2a00)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:15.080689 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252c9cc-b6d8-4a8c-9dbc-ae46b7590f20 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:15.255287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-0252c9cc-b6d8-4a8c-9dbc-ae46b7590f20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee4a78ad41b3453fbf056f453dbb2a00', 'request_id': 'req-0252c9cc-b6d8-4a8c-9dbc-ae46b7590f20'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:15.265264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-0252c9cc-b6d8-4a8c-9dbc-ae46b7590f20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee4a78ad41b3453fbf056f453dbb2a00', 'request_id': 'req-0252c9cc-b6d8-4a8c-9dbc-ae46b7590f20'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:15.281348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-0252c9cc-b6d8-4a8c-9dbc-ae46b7590f20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee4a78ad41b3453fbf056f453dbb2a00', 'request_id': 'req-0252c9cc-b6d8-4a8c-9dbc-ae46b7590f20'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:15.392031 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-0252c9cc-b6d8-4a8c-9dbc-ae46b7590f20 admin admin] The token cache is being invalidate because project ee4a78ad41b3453fbf056f453dbb2a00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:15.392729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-0252c9cc-b6d8-4a8c-9dbc-ae46b7590f20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:15.393226 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-0252c9cc-b6d8-4a8c-9dbc-ae46b7590f20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:15.394863 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1241/2492] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:15 2022] DELETE /identity/v3/projects/ee4a78ad41b3453fbf056f453dbb2a00 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:15.400923 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75201b-50bc-4a83-85d3-cd1596aad384 service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:15.411909 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75201b-50bc-4a83-85d3-cd1596aad384 service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:15.468877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:15.478327 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa625cdf-1070-4b3b-be1d-087102e1c1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:15.480976 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa625cdf-1070-4b3b-be1d-087102e1c1f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:15.480976 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa625cdf-1070-4b3b-be1d-087102e1c1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:15.480976 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa625cdf-1070-4b3b-be1d-087102e1c1f4 admin admin] PATH_INFO: `/v3/users/124462a12cd64ad1abddac93c2b52b7d` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:15.480976 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa625cdf-1070-4b3b-be1d-087102e1c1f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=124462a12cd64ad1abddac93c2b52b7d)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:15.483460 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa625cdf-1070-4b3b-be1d-087102e1c1f4 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:15.671753 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-fa625cdf-1070-4b3b-be1d-087102e1c1f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '124462a12cd64ad1abddac93c2b52b7d'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:15.752190 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-fa625cdf-1070-4b3b-be1d-087102e1c1f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '124462a12cd64ad1abddac93c2b52b7d'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:15.754596 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-fa625cdf-1070-4b3b-be1d-087102e1c1f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '124462a12cd64ad1abddac93c2b52b7d'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:15.772372 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-fa625cdf-1070-4b3b-be1d-087102e1c1f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '124462a12cd64ad1abddac93c2b52b7d'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:15.784194 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-fa625cdf-1070-4b3b-be1d-087102e1c1f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '124462a12cd64ad1abddac93c2b52b7d'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:15.790882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1252/2493] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:15 2022] DELETE /identity/v3/users/124462a12cd64ad1abddac93c2b52b7d => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:16.352781 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252c9cc-b6d8-4a8c-9dbc-ae46b7590f20 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:16.354329 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-0252c9cc-b6d8-4a8c-9dbc-ae46b7590f20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:16.379461 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252c9cc-b6d8-4a8c-9dbc-ae46b7590f20 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:16.458815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c21b11-1a1b-45f1-81e7-4467b306356c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:16.458815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c21b11-1a1b-45f1-81e7-4467b306356c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:16.458815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c21b11-1a1b-45f1-81e7-4467b306356c admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:16.458815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c21b11-1a1b-45f1-81e7-4467b306356c admin admin] PATH_INFO: `/v3/projects/13ab433331e14202aa18a558dd0749b5` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:16.459556 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c21b11-1a1b-45f1-81e7-4467b306356c admin admin] RBAC: Authorizing `identity:delete_project(project_id=13ab433331e14202aa18a558dd0749b5)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:16.459556 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c21b11-1a1b-45f1-81e7-4467b306356c admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:16.650157 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-57c21b11-1a1b-45f1-81e7-4467b306356c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13ab433331e14202aa18a558dd0749b5', 'request_id': 'req-57c21b11-1a1b-45f1-81e7-4467b306356c'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:16.652879 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:16.658179 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-57c21b11-1a1b-45f1-81e7-4467b306356c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13ab433331e14202aa18a558dd0749b5', 'request_id': 'req-57c21b11-1a1b-45f1-81e7-4467b306356c'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:16.665178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-57c21b11-1a1b-45f1-81e7-4467b306356c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13ab433331e14202aa18a558dd0749b5', 'request_id': 'req-57c21b11-1a1b-45f1-81e7-4467b306356c'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:16.780681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-57c21b11-1a1b-45f1-81e7-4467b306356c admin admin] The token cache is being invalidate because project 13ab433331e14202aa18a558dd0749b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:16.781073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-57c21b11-1a1b-45f1-81e7-4467b306356c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:16.781767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-57c21b11-1a1b-45f1-81e7-4467b306356c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:16.783392 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1242/2494] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:16 2022] DELETE /identity/v3/projects/13ab433331e14202aa18a558dd0749b5 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:16.789897 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5d582-d7a5-4036-80ef-f4917693ee2f None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:16.790539 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5d582-d7a5-4036-80ef-f4917693ee2f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:16.790953 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5d582-d7a5-4036-80ef-f4917693ee2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:16.825021 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-e4e5d582-d7a5-4036-80ef-f4917693ee2f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:16.827286 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-e4e5d582-d7a5-4036-80ef-f4917693ee2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:16.911357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1253/2495] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:16.921794 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1728d-18f6-4987-b458-569d7f84b918 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:16.922228 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1728d-18f6-4987-b458-569d7f84b918 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:16.922560 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1728d-18f6-4987-b458-569d7f84b918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:17.094278 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-72c1728d-18f6-4987-b458-569d7f84b918 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:17.095542 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-72c1728d-18f6-4987-b458-569d7f84b918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:17.103654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1243/2496] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:17.108143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5d582-d7a5-4036-80ef-f4917693ee2f None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:17.119131 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5d582-d7a5-4036-80ef-f4917693ee2f None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:17.124321 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d577bb5-2496-4d1b-b2f5-73d09876fa85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:17.124997 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d577bb5-2496-4d1b-b2f5-73d09876fa85 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:17.125272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d577bb5-2496-4d1b-b2f5-73d09876fa85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:17.125551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d577bb5-2496-4d1b-b2f5-73d09876fa85 admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:17.126225 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d577bb5-2496-4d1b-b2f5-73d09876fa85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:17.126531 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d577bb5-2496-4d1b-b2f5-73d09876fa85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:17.128243 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d577bb5-2496-4d1b-b2f5-73d09876fa85 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:17.145091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1254/2497] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:17.155931 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1728d-18f6-4987-b458-569d7f84b918 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:17.183879 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1728d-18f6-4987-b458-569d7f84b918 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:17.186763 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedab5c-23e6-411f-969c-6331d9387f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:17.187633 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedab5c-23e6-411f-969c-6331d9387f67 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:17.188105 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedab5c-23e6-411f-969c-6331d9387f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:17.188472 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedab5c-23e6-411f-969c-6331d9387f67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:17.189282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bedab5c-23e6-411f-969c-6331d9387f67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:17.192568 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bedab5c-23e6-411f-969c-6331d9387f67 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:17.319509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1244/2498] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:17 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:17.324287 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d577bb5-2496-4d1b-b2f5-73d09876fa85 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:17.338805 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d577bb5-2496-4d1b-b2f5-73d09876fa85 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:17.426065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3c08b-20a9-4d95-abec-6cb5356a39d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:17.426718 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3c08b-20a9-4d95-abec-6cb5356a39d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:17.427097 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3c08b-20a9-4d95-abec-6cb5356a39d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:17.427323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3c08b-20a9-4d95-abec-6cb5356a39d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:17.428165 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd3c08b-20a9-4d95-abec-6cb5356a39d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:17.430366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd3c08b-20a9-4d95-abec-6cb5356a39d1 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:17.440228 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:17.615893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1255/2499] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:17 2022] POST /identity/v3/users => generated 290 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:17.620336 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedab5c-23e6-411f-969c-6331d9387f67 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:17.631832 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedab5c-23e6-411f-969c-6331d9387f67 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:17.636904 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9cb36-235d-4a49-bb4b-0cbe87929c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:17.637577 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9cb36-235d-4a49-bb4b-0cbe87929c83 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:17.637901 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9cb36-235d-4a49-bb4b-0cbe87929c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:17.638247 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9cb36-235d-4a49-bb4b-0cbe87929c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:17.638978 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9cb36-235d-4a49-bb4b-0cbe87929c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:17.640892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9cb36-235d-4a49-bb4b-0cbe87929c83 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:17.652234 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1245/2500] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:17.661030 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3c08b-20a9-4d95-abec-6cb5356a39d1 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:17.678492 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3c08b-20a9-4d95-abec-6cb5356a39d1 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:17.683597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d68257-0345-47e9-8481-fba571bfc7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:17.684254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d68257-0345-47e9-8481-fba571bfc7c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:17.684532 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d68257-0345-47e9-8481-fba571bfc7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:17.684946 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d68257-0345-47e9-8481-fba571bfc7c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:17.685732 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d68257-0345-47e9-8481-fba571bfc7c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:17.689165 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d68257-0345-47e9-8481-fba571bfc7c9 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:17.697155 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9cb36-235d-4a49-bb4b-0cbe87929c83 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:17.710464 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-57c9cb36-235d-4a49-bb4b-0cbe87929c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:17.711614 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1256/2501] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:17.712194 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d68257-0345-47e9-8481-fba571bfc7c9 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:17.719453 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9cb36-235d-4a49-bb4b-0cbe87929c83 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:17.725979 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28fa4b2-d03f-44aa-9a8a-be4badcd930b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:17.726878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fa4b2-d03f-44aa-9a8a-be4badcd930b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:17.727161 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fa4b2-d03f-44aa-9a8a-be4badcd930b admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:17.727452 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fa4b2-d03f-44aa-9a8a-be4badcd930b admin admin] PATH_INFO: `/v3/users/a699c03a53e84c5ba311d0700c8e0fc7` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:17.736715 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d68257-0345-47e9-8481-fba571bfc7c9 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:17.736896 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28fa4b2-d03f-44aa-9a8a-be4badcd930b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a699c03a53e84c5ba311d0700c8e0fc7)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:17.740185 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28fa4b2-d03f-44aa-9a8a-be4badcd930b admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:17.743722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e975947-bf5e-43f8-bfbe-238611acb5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:17.745521 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e975947-bf5e-43f8-bfbe-238611acb5e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:17.746540 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e975947-bf5e-43f8-bfbe-238611acb5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:17.747212 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e975947-bf5e-43f8-bfbe-238611acb5e0 admin admin] PATH_INFO: `/v3/projects/f2de56c261804bcb8b740ab5bacfd555/users/03268b6126a842eb99d5c44b93965e92/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:17.770959 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e975947-bf5e-43f8-bfbe-238611acb5e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2de56c261804bcb8b740ab5bacfd555, user_id=03268b6126a842eb99d5c44b93965e92, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:17.776413 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e975947-bf5e-43f8-bfbe-238611acb5e0 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:17.914172 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1257/2502] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:17 2022] PUT /identity/v3/projects/f2de56c261804bcb8b740ab5bacfd555/users/03268b6126a842eb99d5c44b93965e92/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:17.930138 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5059e97-1ee9-4774-b0e4-3ff0dd100712 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:17.930644 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5059e97-1ee9-4774-b0e4-3ff0dd100712 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:17.930954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5059e97-1ee9-4774-b0e4-3ff0dd100712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:17.963352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-c5059e97-1ee9-4774-b0e4-3ff0dd100712 None None] MFA Rules not processed for user `03268b6126a842eb99d5c44b93965e92`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:17.964637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-c5059e97-1ee9-4774-b0e4-3ff0dd100712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:18.040271 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f28fa4b2-d03f-44aa-9a8a-be4badcd930b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a699c03a53e84c5ba311d0700c8e0fc7'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:18.043205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1258/2503] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:18.052013 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5059e97-1ee9-4774-b0e4-3ff0dd100712 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:18.129347 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5059e97-1ee9-4774-b0e4-3ff0dd100712 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:18.134104 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26813c9a-fc1f-4bfb-9cb6-c4dfc9a02b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:18.134847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26813c9a-fc1f-4bfb-9cb6-c4dfc9a02b53 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:18.135127 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26813c9a-fc1f-4bfb-9cb6-c4dfc9a02b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:18.135395 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26813c9a-fc1f-4bfb-9cb6-c4dfc9a02b53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:18.136039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26813c9a-fc1f-4bfb-9cb6-c4dfc9a02b53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:18.137987 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26813c9a-fc1f-4bfb-9cb6-c4dfc9a02b53 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:18.175322 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f28fa4b2-d03f-44aa-9a8a-be4badcd930b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a699c03a53e84c5ba311d0700c8e0fc7'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:18.176238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f28fa4b2-d03f-44aa-9a8a-be4badcd930b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a699c03a53e84c5ba311d0700c8e0fc7'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:18.187536 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f28fa4b2-d03f-44aa-9a8a-be4badcd930b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a699c03a53e84c5ba311d0700c8e0fc7'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:18.188938 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f28fa4b2-d03f-44aa-9a8a-be4badcd930b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a699c03a53e84c5ba311d0700c8e0fc7'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:18.214150 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1246/2504] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:17 2022] DELETE /identity/v3/users/a699c03a53e84c5ba311d0700c8e0fc7 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:18.249179 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a069a2-ecaa-4c9f-92ae-0f25ee2e70d9 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:18.249179 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a069a2-ecaa-4c9f-92ae-0f25ee2e70d9 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:18.249179 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a069a2-ecaa-4c9f-92ae-0f25ee2e70d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:18.249179 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-21a069a2-ecaa-4c9f-92ae-0f25ee2e70d9 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:18.250083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-21a069a2-ecaa-4c9f-92ae-0f25ee2e70d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:18.330074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1247/2505] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:18.339309 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a069a2-ecaa-4c9f-92ae-0f25ee2e70d9 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:18.340772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-21a069a2-ecaa-4c9f-92ae-0f25ee2e70d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:18.361301 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1259/2506] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:18 2022] POST /identity/v3/projects => generated 379 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:18.363247 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:18.368722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26813c9a-fc1f-4bfb-9cb6-c4dfc9a02b53 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:18.372049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-26813c9a-fc1f-4bfb-9cb6-c4dfc9a02b53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:18.416978 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a069a2-ecaa-4c9f-92ae-0f25ee2e70d9 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:18.429418 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:18.469472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26813c9a-fc1f-4bfb-9cb6-c4dfc9a02b53 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:18.474815 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6596743-e051-4527-ba8d-4ed172be9608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:18.475447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6596743-e051-4527-ba8d-4ed172be9608 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:18.475723 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6596743-e051-4527-ba8d-4ed172be9608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:18.475991 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6596743-e051-4527-ba8d-4ed172be9608 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:18.476640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6596743-e051-4527-ba8d-4ed172be9608 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:18.478610 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6596743-e051-4527-ba8d-4ed172be9608 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:18.488789 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b8b05-a043-488e-ae12-2dc56a5da114 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:18.491341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b8b05-a043-488e-ae12-2dc56a5da114 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:18.491341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b8b05-a043-488e-ae12-2dc56a5da114 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:18.491341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b8b05-a043-488e-ae12-2dc56a5da114 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:18.503029 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b8b05-a043-488e-ae12-2dc56a5da114 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:18.504591 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b8b05-a043-488e-ae12-2dc56a5da114 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:18.515121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1248/2507] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:18.607108 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1260/2508] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:18 2022] POST /identity/v3/users => generated 290 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:18.615090 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b8b05-a043-488e-ae12-2dc56a5da114 service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:18.628515 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b8b05-a043-488e-ae12-2dc56a5da114 service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:18.644355 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abcbf8-6672-4fb0-94ca-3b2e8e243aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:18.645029 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abcbf8-6672-4fb0-94ca-3b2e8e243aff admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:18.645312 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abcbf8-6672-4fb0-94ca-3b2e8e243aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:18.645579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abcbf8-6672-4fb0-94ca-3b2e8e243aff admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:18.646275 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abcbf8-6672-4fb0-94ca-3b2e8e243aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:18.647977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abcbf8-6672-4fb0-94ca-3b2e8e243aff admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:18.659273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1249/2509] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:18.664582 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6596743-e051-4527-ba8d-4ed172be9608 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:18.676010 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6596743-e051-4527-ba8d-4ed172be9608 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:18.680860 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b746a28-3b67-4cb8-9e0d-d98adfa273de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:18.681517 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b746a28-3b67-4cb8-9e0d-d98adfa273de admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:18.681792 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b746a28-3b67-4cb8-9e0d-d98adfa273de admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:18.682189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b746a28-3b67-4cb8-9e0d-d98adfa273de admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:18.682781 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b746a28-3b67-4cb8-9e0d-d98adfa273de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:18.684431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b746a28-3b67-4cb8-9e0d-d98adfa273de admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:18.697099 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1261/2510] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:18.703493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abcbf8-6672-4fb0-94ca-3b2e8e243aff admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:18.714827 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abcbf8-6672-4fb0-94ca-3b2e8e243aff admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:18.720689 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a3783-7df9-4eeb-b97f-2b281875524c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:18.721370 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a3783-7df9-4eeb-b97f-2b281875524c admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:18.721644 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a3783-7df9-4eeb-b97f-2b281875524c admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:18.722282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a3783-7df9-4eeb-b97f-2b281875524c admin admin] PATH_INFO: `/v3/projects/ffcbb561522a4367af3bde8aabb87c64/users/5b7d845892b844ae81dca93940636ec7/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:18.739806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915a3783-7df9-4eeb-b97f-2b281875524c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffcbb561522a4367af3bde8aabb87c64, user_id=5b7d845892b844ae81dca93940636ec7, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:18.742687 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915a3783-7df9-4eeb-b97f-2b281875524c admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:18.840206 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1250/2511] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:18 2022] PUT /identity/v3/projects/ffcbb561522a4367af3bde8aabb87c64/users/5b7d845892b844ae81dca93940636ec7/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:18.861273 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc99d13-134b-4bf4-bf6c-4f220790b4fc None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:18.861657 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc99d13-134b-4bf4-bf6c-4f220790b4fc None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:18.862064 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc99d13-134b-4bf4-bf6c-4f220790b4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:18.965151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a3783-7df9-4eeb-b97f-2b281875524c admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:18.969454 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-4dc99d13-134b-4bf4-bf6c-4f220790b4fc None None] MFA Rules not processed for user `5b7d845892b844ae81dca93940636ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:18.970279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-4dc99d13-134b-4bf4-bf6c-4f220790b4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:19.100846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a3783-7df9-4eeb-b97f-2b281875524c admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:19.104104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c3171-dc08-49fb-8b81-26fdfc8bd7c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:19.105349 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c3171-dc08-49fb-8b81-26fdfc8bd7c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:19.105888 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c3171-dc08-49fb-8b81-26fdfc8bd7c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:19.106413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c3171-dc08-49fb-8b81-26fdfc8bd7c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:19.109935 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-281c3171-dc08-49fb-8b81-26fdfc8bd7c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:19.143589 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1262/2512] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:19.148692 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc99d13-134b-4bf4-bf6c-4f220790b4fc None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:19.211842 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:19.240651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c3171-dc08-49fb-8b81-26fdfc8bd7c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:19.242344 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c3171-dc08-49fb-8b81-26fdfc8bd7c7 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:19.255957 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1251/2513] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:19.256278 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c3171-dc08-49fb-8b81-26fdfc8bd7c7 service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:19.257837 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-281c3171-dc08-49fb-8b81-26fdfc8bd7c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:19.277506 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c3171-dc08-49fb-8b81-26fdfc8bd7c7 service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:19.283956 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0828d2-e9c5-487b-8417-3a7d1e597609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:19.284834 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0828d2-e9c5-487b-8417-3a7d1e597609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:19.285212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0828d2-e9c5-487b-8417-3a7d1e597609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:19.285585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0828d2-e9c5-487b-8417-3a7d1e597609 admin admin] PATH_INFO: `/v3/projects/df87dc04301c41bca9a33febf716f62d` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:19.287006 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0828d2-e9c5-487b-8417-3a7d1e597609 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df87dc04301c41bca9a33febf716f62d)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:19.289553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0828d2-e9c5-487b-8417-3a7d1e597609 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:19.339918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc99d13-134b-4bf4-bf6c-4f220790b4fc None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:19.350110 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7480641-fa28-44fb-84b1-24c925d3ed3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:19.351549 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7480641-fa28-44fb-84b1-24c925d3ed3f admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:19.351927 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7480641-fa28-44fb-84b1-24c925d3ed3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:19.353297 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7480641-fa28-44fb-84b1-24c925d3ed3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:19.353640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7480641-fa28-44fb-84b1-24c925d3ed3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:19.357982 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7480641-fa28-44fb-84b1-24c925d3ed3f admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:19.434508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-df0828d2-e9c5-487b-8417-3a7d1e597609 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df87dc04301c41bca9a33febf716f62d', 'request_id': 'req-df0828d2-e9c5-487b-8417-3a7d1e597609'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:19.447200 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-df0828d2-e9c5-487b-8417-3a7d1e597609 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df87dc04301c41bca9a33febf716f62d', 'request_id': 'req-df0828d2-e9c5-487b-8417-3a7d1e597609'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:19.454762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-df0828d2-e9c5-487b-8417-3a7d1e597609 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df87dc04301c41bca9a33febf716f62d', 'request_id': 'req-df0828d2-e9c5-487b-8417-3a7d1e597609'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:19.496854 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1263/2514] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:19 2022] POST /identity/v3/projects => generated 379 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:19.502525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7480641-fa28-44fb-84b1-24c925d3ed3f admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:19.539230 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:19.588151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7480641-fa28-44fb-84b1-24c925d3ed3f admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:19.594708 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890a53f-3fa5-4731-9e39-12fdd8c68ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:19.595354 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890a53f-3fa5-4731-9e39-12fdd8c68ade admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:19.595627 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890a53f-3fa5-4731-9e39-12fdd8c68ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:19.595900 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890a53f-3fa5-4731-9e39-12fdd8c68ade admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:19.596564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4890a53f-3fa5-4731-9e39-12fdd8c68ade admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:19.598667 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4890a53f-3fa5-4731-9e39-12fdd8c68ade admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:19.615552 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-df0828d2-e9c5-487b-8417-3a7d1e597609 admin admin] The token cache is being invalidate because project df87dc04301c41bca9a33febf716f62d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:19.616136 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-df0828d2-e9c5-487b-8417-3a7d1e597609 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:19.617113 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-df0828d2-e9c5-487b-8417-3a7d1e597609 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:19.619064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1252/2515] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:19 2022] DELETE /identity/v3/projects/df87dc04301c41bca9a33febf716f62d => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:19.625203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aef2ad-eb6a-4b22-a8b3-bf2d04b16985 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:19.625611 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aef2ad-eb6a-4b22-a8b3-bf2d04b16985 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:19.625978 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aef2ad-eb6a-4b22-a8b3-bf2d04b16985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:19.659891 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-41aef2ad-eb6a-4b22-a8b3-bf2d04b16985 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:19.662426 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-41aef2ad-eb6a-4b22-a8b3-bf2d04b16985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:19.674839 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1253/2516] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:19.689342 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6c65c-fcb2-4c74-9b29-c65dc252366c None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:19.689817 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6c65c-fcb2-4c74-9b29-c65dc252366c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:19.690268 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6c65c-fcb2-4c74-9b29-c65dc252366c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:19.717042 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-2ed6c65c-fcb2-4c74-9b29-c65dc252366c None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:19.718458 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-2ed6c65c-fcb2-4c74-9b29-c65dc252366c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:19.729552 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1254/2517] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:19.734788 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1264/2518] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:19 2022] POST /identity/v3/users => generated 296 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:19.736079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6c65c-fcb2-4c74-9b29-c65dc252366c None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:19.738638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890a53f-3fa5-4731-9e39-12fdd8c68ade admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:19.744207 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-4890a53f-3fa5-4731-9e39-12fdd8c68ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:19.752493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6c65c-fcb2-4c74-9b29-c65dc252366c None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:19.760517 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890a53f-3fa5-4731-9e39-12fdd8c68ade admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:19.762552 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07018b49-2e1f-4759-8b9b-ce785f724d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:19.764328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07018b49-2e1f-4759-8b9b-ce785f724d9f admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:19.764748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07018b49-2e1f-4759-8b9b-ce785f724d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:19.765196 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07018b49-2e1f-4759-8b9b-ce785f724d9f admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:19.766308 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07018b49-2e1f-4759-8b9b-ce785f724d9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:19.766830 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07018b49-2e1f-4759-8b9b-ce785f724d9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:19.768778 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc76e21-2f18-4512-8349-79f0fe642c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:19.769659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc76e21-2f18-4512-8349-79f0fe642c7e admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:19.770024 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07018b49-2e1f-4759-8b9b-ce785f724d9f admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:19.770299 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc76e21-2f18-4512-8349-79f0fe642c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:19.770597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc76e21-2f18-4512-8349-79f0fe642c7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:19.771380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc76e21-2f18-4512-8349-79f0fe642c7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:19.773219 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc76e21-2f18-4512-8349-79f0fe642c7e admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:19.787688 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1265/2519] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:19.788194 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1255/2520] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:01:19.792976 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07018b49-2e1f-4759-8b9b-ce785f724d9f admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:19.793192 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc76e21-2f18-4512-8349-79f0fe642c7e admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:19.804596 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc76e21-2f18-4512-8349-79f0fe642c7e admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:19.804862 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07018b49-2e1f-4759-8b9b-ce785f724d9f admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:19.809898 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7513da-837c-4f48-b60e-36ff8e032371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:19.811859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7513da-837c-4f48-b60e-36ff8e032371 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:19.812196 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7513da-837c-4f48-b60e-36ff8e032371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:19.812465 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7513da-837c-4f48-b60e-36ff8e032371 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:19.816178 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dcfabe-1256-4e58-ab64-252fe6fb32d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:19.816390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7513da-837c-4f48-b60e-36ff8e032371 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:19.818835 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcfabe-1256-4e58-ab64-252fe6fb32d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:19.818835 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcfabe-1256-4e58-ab64-252fe6fb32d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:19.818835 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcfabe-1256-4e58-ab64-252fe6fb32d4 admin admin] PATH_INFO: `/v3/projects/0abf2bede458497789f8e7c50014873f/users/7be6501df0c84b1c9dd0e87dd67e21f3/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:19.819313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7513da-837c-4f48-b60e-36ff8e032371 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:19.848179 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dcfabe-1256-4e58-ab64-252fe6fb32d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0abf2bede458497789f8e7c50014873f, user_id=7be6501df0c84b1c9dd0e87dd67e21f3, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:19.853804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dcfabe-1256-4e58-ab64-252fe6fb32d4 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:19.956075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1256/2522] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:19 2022] PUT /identity/v3/projects/0abf2bede458497789f8e7c50014873f/users/7be6501df0c84b1c9dd0e87dd67e21f3/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:19.960449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1266/2522] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:19 2022] POST /identity/v3/projects => generated 369 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:19.960768 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dcfabe-1256-4e58-ab64-252fe6fb32d4 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:19.962766 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7513da-837c-4f48-b60e-36ff8e032371 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:20.075752 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7513da-837c-4f48-b60e-36ff8e032371 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:20.082979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e828fb-4d03-4680-8757-df46fc9cd090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:20.083898 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e828fb-4d03-4680-8757-df46fc9cd090 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:20.084365 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e828fb-4d03-4680-8757-df46fc9cd090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:20.084817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e828fb-4d03-4680-8757-df46fc9cd090 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:20.085034 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dcfabe-1256-4e58-ab64-252fe6fb32d4 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:20.085813 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e828fb-4d03-4680-8757-df46fc9cd090 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:20.088267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e828fb-4d03-4680-8757-df46fc9cd090 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:20.092465 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5c6c0-dfa8-4b4f-b987-f03f936c5731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:20.093355 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5c6c0-dfa8-4b4f-b987-f03f936c5731 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:20.093760 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5c6c0-dfa8-4b4f-b987-f03f936c5731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:20.094153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5c6c0-dfa8-4b4f-b987-f03f936c5731 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:20.095299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5c6c0-dfa8-4b4f-b987-f03f936c5731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:20.097990 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5c6c0-dfa8-4b4f-b987-f03f936c5731 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:20.126872 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1257/2523] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:20.126872 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5c6c0-dfa8-4b4f-b987-f03f936c5731 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:20.137220 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5c6c0-dfa8-4b4f-b987-f03f936c5731 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:20.145248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a9a07-9ad1-4cfe-b06d-45fe62c7cca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:20.146337 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a9a07-9ad1-4cfe-b06d-45fe62c7cca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:20.146821 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a9a07-9ad1-4cfe-b06d-45fe62c7cca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:20.147234 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a9a07-9ad1-4cfe-b06d-45fe62c7cca2 admin admin] PATH_INFO: `/v3/projects/0abf2bede458497789f8e7c50014873f/users/7be6501df0c84b1c9dd0e87dd67e21f3/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:20.150431 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a9a07-9ad1-4cfe-b06d-45fe62c7cca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0abf2bede458497789f8e7c50014873f, user_id=7be6501df0c84b1c9dd0e87dd67e21f3, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:20.155631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a9a07-9ad1-4cfe-b06d-45fe62c7cca2 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:20.178797 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1258/2524] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:20 2022] PUT /identity/v3/projects/0abf2bede458497789f8e7c50014873f/users/7be6501df0c84b1c9dd0e87dd67e21f3/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:20.196934 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843136b2-1e8c-461a-a302-0e0972ec9e52 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:20.197209 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843136b2-1e8c-461a-a302-0e0972ec9e52 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:20.197567 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843136b2-1e8c-461a-a302-0e0972ec9e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:20.209619 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:20.250086 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-843136b2-1e8c-461a-a302-0e0972ec9e52 None None] MFA Rules not processed for user `7be6501df0c84b1c9dd0e87dd67e21f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:20.252766 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-843136b2-1e8c-461a-a302-0e0972ec9e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:20.268690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1267/2525] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:19 2022] POST /identity/v3/users => generated 285 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:20.270481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e828fb-4d03-4680-8757-df46fc9cd090 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:20.291358 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e828fb-4d03-4680-8757-df46fc9cd090 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:20.436520 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1259/2526] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:20.455927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843136b2-1e8c-461a-a302-0e0972ec9e52 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:20.458102 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-843136b2-1e8c-461a-a302-0e0972ec9e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:20.463525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a134d702-a146-454a-8abd-8aaeb9fe9bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:20.464042 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134d702-a146-454a-8abd-8aaeb9fe9bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:20.465065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134d702-a146-454a-8abd-8aaeb9fe9bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:20.466520 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134d702-a146-454a-8abd-8aaeb9fe9bf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:20.467065 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a134d702-a146-454a-8abd-8aaeb9fe9bf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:20.470384 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a134d702-a146-454a-8abd-8aaeb9fe9bf2 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:20.489317 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1268/2527] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:20.496078 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a134d702-a146-454a-8abd-8aaeb9fe9bf2 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:20.506969 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:20.514837 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a134d702-a146-454a-8abd-8aaeb9fe9bf2 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:20.519490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216021b9-77c7-4ca0-b26d-68ac763e3320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:20.520256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216021b9-77c7-4ca0-b26d-68ac763e3320 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:20.520639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216021b9-77c7-4ca0-b26d-68ac763e3320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:20.520883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216021b9-77c7-4ca0-b26d-68ac763e3320 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:20.522387 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216021b9-77c7-4ca0-b26d-68ac763e3320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:20.523228 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216021b9-77c7-4ca0-b26d-68ac763e3320 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:20.539152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1269/2528] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:20.552351 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216021b9-77c7-4ca0-b26d-68ac763e3320 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:20.577988 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216021b9-77c7-4ca0-b26d-68ac763e3320 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:20.586914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92573f-a0cd-4cfd-8e20-211a769bb2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:20.588229 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92573f-a0cd-4cfd-8e20-211a769bb2e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:20.588877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92573f-a0cd-4cfd-8e20-211a769bb2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:20.589435 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92573f-a0cd-4cfd-8e20-211a769bb2e2 admin admin] PATH_INFO: `/v3/projects/c26bb29455ee4331ae95bce891d66bdc/users/b15ebc8995444db3a2a0dd89fe9bc6c4/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:20.591580 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843136b2-1e8c-461a-a302-0e0972ec9e52 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:20.600437 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d30b6-c401-4193-9143-86468758fa1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:20.601513 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d30b6-c401-4193-9143-86468758fa1e service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:20.602153 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d30b6-c401-4193-9143-86468758fa1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:20.602679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d30b6-c401-4193-9143-86468758fa1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:20.620844 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be92573f-a0cd-4cfd-8e20-211a769bb2e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c26bb29455ee4331ae95bce891d66bdc, user_id=b15ebc8995444db3a2a0dd89fe9bc6c4, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:20.621472 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5d30b6-c401-4193-9143-86468758fa1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:20.624856 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5d30b6-c401-4193-9143-86468758fa1e service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:20.626369 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be92573f-a0cd-4cfd-8e20-211a769bb2e2 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:20.641062 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1260/2529] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:20 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:20.759687 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1270/2530] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:20 2022] PUT /identity/v3/projects/c26bb29455ee4331ae95bce891d66bdc/users/b15ebc8995444db3a2a0dd89fe9bc6c4/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:20.832026 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d30b6-c401-4193-9143-86468758fa1e service cinder] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:20.836158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-7e5d30b6-c401-4193-9143-86468758fa1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:20.866787 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7321895-aba7-4b87-9f3b-b65ab78448d8 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:20.867265 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7321895-aba7-4b87-9f3b-b65ab78448d8 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:20.867738 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7321895-aba7-4b87-9f3b-b65ab78448d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:20.878739 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d30b6-c401-4193-9143-86468758fa1e service cinder] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:20.943566 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-d7321895-aba7-4b87-9f3b-b65ab78448d8 None None] MFA Rules not processed for user `b15ebc8995444db3a2a0dd89fe9bc6c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:20.946128 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-d7321895-aba7-4b87-9f3b-b65ab78448d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:21.014232 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b99e8a-1acd-419d-8c75-a306cc9939c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:21.017169 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b99e8a-1acd-419d-8c75-a306cc9939c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:21.018236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b99e8a-1acd-419d-8c75-a306cc9939c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:21.019778 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b99e8a-1acd-419d-8c75-a306cc9939c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:21.028444 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-38b99e8a-1acd-419d-8c75-a306cc9939c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:21.236168 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:21.269503 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b99e8a-1acd-419d-8c75-a306cc9939c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:21.278456 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1271/2531] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:21.291797 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b99e8a-1acd-419d-8c75-a306cc9939c6 service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:21.299056 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7321895-aba7-4b87-9f3b-b65ab78448d8 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:21.305414 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-d7321895-aba7-4b87-9f3b-b65ab78448d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:21.309915 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1261/2532] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:21.445719 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7321895-aba7-4b87-9f3b-b65ab78448d8 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:21.452795 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf149a04-631b-49c7-aced-dcb103d9efdf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea51ef02128a4b63b11049eb0daef298', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:21.454007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf149a04-631b-49c7-aced-dcb103d9efdf service glance] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:21.458023 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf149a04-631b-49c7-aced-dcb103d9efdf service glance] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:21.458023 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf149a04-631b-49c7-aced-dcb103d9efdf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:21.472047 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf149a04-631b-49c7-aced-dcb103d9efdf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:21.474895 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf149a04-631b-49c7-aced-dcb103d9efdf service glance] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:21.494006 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1272/2533] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:24.822915 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e727cfd-233e-4a00-bbba-cf9d2c484811 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:24.823371 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e727cfd-233e-4a00-bbba-cf9d2c484811 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:24.823460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e727cfd-233e-4a00-bbba-cf9d2c484811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:24.864219 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-2e727cfd-233e-4a00-bbba-cf9d2c484811 None None] MFA Rules not processed for user `9646662b0b3048f79ad5d2b9602f10a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:24.865425 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-2e727cfd-233e-4a00-bbba-cf9d2c484811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:24.932288 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1262/2534] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Aug 8 17:01:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:24.947382 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf149a04-631b-49c7-aced-dcb103d9efdf service glance] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:24.949026 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-cf149a04-631b-49c7-aced-dcb103d9efdf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:24.962945 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e727cfd-233e-4a00-bbba-cf9d2c484811 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:24.968131 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-2e727cfd-233e-4a00-bbba-cf9d2c484811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:25.131429 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf149a04-631b-49c7-aced-dcb103d9efdf service glance] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:25.143701 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:25.157781 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c43720-7835-4ca2-a3b8-39191359b426 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c606decb832444e38a194b98f8d15f8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:25.158547 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c43720-7835-4ca2-a3b8-39191359b426 service swift] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:25.160994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c43720-7835-4ca2-a3b8-39191359b426 service swift] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:25.161342 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c43720-7835-4ca2-a3b8-39191359b426 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:25.179750 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e727cfd-233e-4a00-bbba-cf9d2c484811 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:25.183981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2e4ac-477e-4866-a23e-e6631565eeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:25.184796 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2e4ac-477e-4866-a23e-e6631565eeba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:25.185084 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2e4ac-477e-4866-a23e-e6631565eeba admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:25.186336 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2e4ac-477e-4866-a23e-e6631565eeba admin admin] PATH_INFO: `/v3/users/fdb16e7edbe143e9b96325842896f04e` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:25.189331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c43720-7835-4ca2-a3b8-39191359b426 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:25.191269 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2e4ac-477e-4866-a23e-e6631565eeba admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdb16e7edbe143e9b96325842896f04e)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:25.193301 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2e4ac-477e-4866-a23e-e6631565eeba admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:25.194023 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c43720-7835-4ca2-a3b8-39191359b426 service swift] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:25.239432 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1273/2535] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Aug 8 17:01:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 17:01:25.508464 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-a8e2e4ac-477e-4866-a23e-e6631565eeba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdb16e7edbe143e9b96325842896f04e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:25.627056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-a8e2e4ac-477e-4866-a23e-e6631565eeba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdb16e7edbe143e9b96325842896f04e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:25.628064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-a8e2e4ac-477e-4866-a23e-e6631565eeba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdb16e7edbe143e9b96325842896f04e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:25.640773 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-a8e2e4ac-477e-4866-a23e-e6631565eeba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdb16e7edbe143e9b96325842896f04e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:25.642048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-a8e2e4ac-477e-4866-a23e-e6631565eeba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdb16e7edbe143e9b96325842896f04e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:25.684369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1263/2536] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:24 2022] DELETE /identity/v3/users/fdb16e7edbe143e9b96325842896f04e => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:26.057867 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c43720-7835-4ca2-a3b8-39191359b426 service swift] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:26.072107 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c43720-7835-4ca2-a3b8-39191359b426 service swift] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:26.208477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d655e2d-fb47-4285-a42c-4661a70ce095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:26.209688 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d655e2d-fb47-4285-a42c-4661a70ce095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:26.210212 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d655e2d-fb47-4285-a42c-4661a70ce095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:26.210838 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d655e2d-fb47-4285-a42c-4661a70ce095 admin admin] PATH_INFO: `/v3/projects/b2063a114c1942cab80ae4b708be9ab2` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:26.213277 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d655e2d-fb47-4285-a42c-4661a70ce095 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2063a114c1942cab80ae4b708be9ab2)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:26.217494 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d655e2d-fb47-4285-a42c-4661a70ce095 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:26.411536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-4d655e2d-fb47-4285-a42c-4661a70ce095 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2063a114c1942cab80ae4b708be9ab2', 'request_id': 'req-4d655e2d-fb47-4285-a42c-4661a70ce095'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:26.498744 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-4d655e2d-fb47-4285-a42c-4661a70ce095 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2063a114c1942cab80ae4b708be9ab2', 'request_id': 'req-4d655e2d-fb47-4285-a42c-4661a70ce095'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:26.502683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2e4ac-477e-4866-a23e-e6631565eeba admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:26.506441 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-a8e2e4ac-477e-4866-a23e-e6631565eeba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:26.522482 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-4d655e2d-fb47-4285-a42c-4661a70ce095 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2063a114c1942cab80ae4b708be9ab2', 'request_id': 'req-4d655e2d-fb47-4285-a42c-4661a70ce095'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:26.543873 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-4d655e2d-fb47-4285-a42c-4661a70ce095 admin admin] The token cache is being invalidate because project b2063a114c1942cab80ae4b708be9ab2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:26.544263 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-4d655e2d-fb47-4285-a42c-4661a70ce095 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:26.545974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-4d655e2d-fb47-4285-a42c-4661a70ce095 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:26.554846 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1274/2537] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:26 2022] DELETE /identity/v3/projects/b2063a114c1942cab80ae4b708be9ab2 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:26.554846 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e3e0a-3f67-466c-9e8a-ace801920026 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:26.554846 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e3e0a-3f67-466c-9e8a-ace801920026 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:26.554846 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e3e0a-3f67-466c-9e8a-ace801920026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:26.559583 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2e4ac-477e-4866-a23e-e6631565eeba admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:26.579534 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:26.587354 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-ff0e3e0a-3f67-466c-9e8a-ace801920026 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:26.588949 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ff0e3e0a-3f67-466c-9e8a-ace801920026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:26.689352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1275/2538] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:26.692565 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5d960-240b-4b6e-82a1-4dc34cc1f963 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:26.693119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5d960-240b-4b6e-82a1-4dc34cc1f963 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:26.693603 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5d960-240b-4b6e-82a1-4dc34cc1f963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:26.700408 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43297dbc-9a9a-4e4a-96df-7292e2c4f0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:26.702627 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43297dbc-9a9a-4e4a-96df-7292e2c4f0ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:26.702627 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43297dbc-9a9a-4e4a-96df-7292e2c4f0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:26.703035 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43297dbc-9a9a-4e4a-96df-7292e2c4f0ea admin admin] PATH_INFO: `/v3/users/03268b6126a842eb99d5c44b93965e92` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:26.704165 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43297dbc-9a9a-4e4a-96df-7292e2c4f0ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=03268b6126a842eb99d5c44b93965e92)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:26.706452 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43297dbc-9a9a-4e4a-96df-7292e2c4f0ea admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:26.758637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-08a5d960-240b-4b6e-82a1-4dc34cc1f963 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:26.758637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-08a5d960-240b-4b6e-82a1-4dc34cc1f963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:26.769761 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1276/2539] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:26.776833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5d960-240b-4b6e-82a1-4dc34cc1f963 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:26.789643 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5d960-240b-4b6e-82a1-4dc34cc1f963 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:26.797421 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b6947-b152-45d7-a784-d4af9e4d28e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:26.798088 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b6947-b152-45d7-a784-d4af9e4d28e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:26.798373 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b6947-b152-45d7-a784-d4af9e4d28e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:26.801347 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b6947-b152-45d7-a784-d4af9e4d28e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:26.802195 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b6947-b152-45d7-a784-d4af9e4d28e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:26.802764 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b6947-b152-45d7-a784-d4af9e4d28e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:26.804459 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b6947-b152-45d7-a784-d4af9e4d28e2 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:26.821713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1277/2540] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:01:26.827280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b6947-b152-45d7-a784-d4af9e4d28e2 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:26.834407 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:26.842646 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b6947-b152-45d7-a784-d4af9e4d28e2 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:26.851705 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd0324-5181-4f6b-8857-2f1339a39801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:26.852759 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd0324-5181-4f6b-8857-2f1339a39801 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:26.853290 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd0324-5181-4f6b-8857-2f1339a39801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:26.853996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd0324-5181-4f6b-8857-2f1339a39801 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:26.855357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd0324-5181-4f6b-8857-2f1339a39801 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:26.860953 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd0324-5181-4f6b-8857-2f1339a39801 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:26.961752 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-43297dbc-9a9a-4e4a-96df-7292e2c4f0ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03268b6126a842eb99d5c44b93965e92'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:26.978667 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1278/2541] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:26 2022] POST /identity/v3/projects => generated 405 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:26.984075 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd0324-5181-4f6b-8857-2f1339a39801 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:27.053589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-43297dbc-9a9a-4e4a-96df-7292e2c4f0ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03268b6126a842eb99d5c44b93965e92'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:27.054488 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-43297dbc-9a9a-4e4a-96df-7292e2c4f0ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03268b6126a842eb99d5c44b93965e92'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:27.066974 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-43297dbc-9a9a-4e4a-96df-7292e2c4f0ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03268b6126a842eb99d5c44b93965e92'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:27.068448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-43297dbc-9a9a-4e4a-96df-7292e2c4f0ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03268b6126a842eb99d5c44b93965e92'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:27.097460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1264/2542] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:26 2022] DELETE /identity/v3/users/03268b6126a842eb99d5c44b93965e92 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:27.102842 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5911a7-f13e-4713-8ff1-f5ecc5ae846e None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:27.103392 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5911a7-f13e-4713-8ff1-f5ecc5ae846e None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:27.103849 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5911a7-f13e-4713-8ff1-f5ecc5ae846e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:27.119474 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd0324-5181-4f6b-8857-2f1339a39801 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:27.132726 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-ff5911a7-f13e-4713-8ff1-f5ecc5ae846e None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:27.135166 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ff5911a7-f13e-4713-8ff1-f5ecc5ae846e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:27.254352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25071b86-34b3-4641-aef7-31b65ea9123d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:27.255038 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25071b86-34b3-4641-aef7-31b65ea9123d admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:27.255384 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25071b86-34b3-4641-aef7-31b65ea9123d admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:27.255719 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25071b86-34b3-4641-aef7-31b65ea9123d admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:27.257747 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25071b86-34b3-4641-aef7-31b65ea9123d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:27.259910 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25071b86-34b3-4641-aef7-31b65ea9123d admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:27.319684 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1265/2543] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:27.331473 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5911a7-f13e-4713-8ff1-f5ecc5ae846e None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:27.333817 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ff5911a7-f13e-4713-8ff1-f5ecc5ae846e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:27.390721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1279/2544] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:26 2022] POST /identity/v3/users => generated 303 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:27.395910 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25071b86-34b3-4641-aef7-31b65ea9123d admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:27.400259 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-25071b86-34b3-4641-aef7-31b65ea9123d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:27.437675 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25071b86-34b3-4641-aef7-31b65ea9123d admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:27.443808 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c9e172-1530-442f-a26c-f229362dcaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:27.444502 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9e172-1530-442f-a26c-f229362dcaf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:27.444816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9e172-1530-442f-a26c-f229362dcaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:27.445069 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9e172-1530-442f-a26c-f229362dcaf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:27.445740 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c9e172-1530-442f-a26c-f229362dcaf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:27.447461 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c9e172-1530-442f-a26c-f229362dcaf4 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:27.463403 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1280/2545] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:27.467400 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c9e172-1530-442f-a26c-f229362dcaf4 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:27.469152 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:27.497342 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c9e172-1530-442f-a26c-f229362dcaf4 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:27.516675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5911a7-f13e-4713-8ff1-f5ecc5ae846e None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:27.519531 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36726219-3131-42e0-b32c-41ba3d761d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:27.520468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36726219-3131-42e0-b32c-41ba3d761d19 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:27.523448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3c5c5-7235-40ec-986b-a1561672c4d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:27.525650 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3c5c5-7235-40ec-986b-a1561672c4d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:27.526190 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3c5c5-7235-40ec-986b-a1561672c4d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:27.526739 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3c5c5-7235-40ec-986b-a1561672c4d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:27.530769 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36726219-3131-42e0-b32c-41ba3d761d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:27.537796 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36726219-3131-42e0-b32c-41ba3d761d19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:27.539095 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36726219-3131-42e0-b32c-41ba3d761d19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:27.541019 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3c5c5-7235-40ec-986b-a1561672c4d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:27.543721 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3c5c5-7235-40ec-986b-a1561672c4d7 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:27.550928 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36726219-3131-42e0-b32c-41ba3d761d19 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:27.565912 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1266/2546] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:27.581907 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1281/2547] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:27.586295 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3c5c5-7235-40ec-986b-a1561672c4d7 service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:27.599875 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3c5c5-7235-40ec-986b-a1561672c4d7 service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:27.606283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc583d-b099-48a6-8418-f8664f3ae33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:27.607814 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc583d-b099-48a6-8418-f8664f3ae33f admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:27.608083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc583d-b099-48a6-8418-f8664f3ae33f admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:27.608347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc583d-b099-48a6-8418-f8664f3ae33f admin admin] PATH_INFO: `/v3/projects/d336332be1b54b11983c68924779945e/users/59c7b8d09b5d4beaad0ad6430a80eda3/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:27.629527 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc583d-b099-48a6-8418-f8664f3ae33f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d336332be1b54b11983c68924779945e, user_id=59c7b8d09b5d4beaad0ad6430a80eda3, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:27.632271 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc583d-b099-48a6-8418-f8664f3ae33f admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:27.721422 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1267/2548] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:27 2022] PUT /identity/v3/projects/d336332be1b54b11983c68924779945e/users/59c7b8d09b5d4beaad0ad6430a80eda3/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:27.739143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935f892-8d06-42ec-9304-5cf4bf8a8501 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:27.739518 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935f892-8d06-42ec-9304-5cf4bf8a8501 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:27.739810 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935f892-8d06-42ec-9304-5cf4bf8a8501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:27.775674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-3935f892-8d06-42ec-9304-5cf4bf8a8501 None None] MFA Rules not processed for user `59c7b8d09b5d4beaad0ad6430a80eda3`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:27.776986 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-3935f892-8d06-42ec-9304-5cf4bf8a8501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:27.850975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:27.904324 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc583d-b099-48a6-8418-f8664f3ae33f admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:27.905995 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1282/2549] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:27 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:28.095295 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc583d-b099-48a6-8418-f8664f3ae33f admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:28.109793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848c505-3855-4041-a748-55a2d1765eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:28.111022 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848c505-3855-4041-a748-55a2d1765eeb admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:28.111477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848c505-3855-4041-a748-55a2d1765eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:28.112065 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848c505-3855-4041-a748-55a2d1765eeb admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:28.115592 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848c505-3855-4041-a748-55a2d1765eeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:28.119701 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848c505-3855-4041-a748-55a2d1765eeb admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:28.278528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1268/2550] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:27 2022] POST /identity/v3/projects => generated 403 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:28.281037 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935f892-8d06-42ec-9304-5cf4bf8a8501 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:28.313574 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935f892-8d06-42ec-9304-5cf4bf8a8501 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:28.466398 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ceac92-123a-472d-a153-7e56e7f523cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:28.468013 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ceac92-123a-472d-a153-7e56e7f523cc admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:28.468952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ceac92-123a-472d-a153-7e56e7f523cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:28.469673 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ceac92-123a-472d-a153-7e56e7f523cc admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:28.471286 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ceac92-123a-472d-a153-7e56e7f523cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:28.474467 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ceac92-123a-472d-a153-7e56e7f523cc admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:28.558881 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848c505-3855-4041-a748-55a2d1765eeb admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:28.561620 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-8848c505-3855-4041-a748-55a2d1765eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:28.598788 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848c505-3855-4041-a748-55a2d1765eeb admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:28.607990 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dba65f-e38a-4fb5-9242-f8ce85e621f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:28.609071 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dba65f-e38a-4fb5-9242-f8ce85e621f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:28.609411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dba65f-e38a-4fb5-9242-f8ce85e621f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:28.609563 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dba65f-e38a-4fb5-9242-f8ce85e621f9 admin admin] PATH_INFO: `/v3/projects/f2de56c261804bcb8b740ab5bacfd555` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:28.610958 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dba65f-e38a-4fb5-9242-f8ce85e621f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2de56c261804bcb8b740ab5bacfd555)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:28.613576 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dba65f-e38a-4fb5-9242-f8ce85e621f9 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:28.614841 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1283/2551] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:28 2022] POST /identity/v3/users => generated 308 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:28.619039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ceac92-123a-472d-a153-7e56e7f523cc admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:28.633192 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ceac92-123a-472d-a153-7e56e7f523cc admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:28.639815 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93895fa-62ff-4d79-9958-813b51b9d104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:28.640586 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93895fa-62ff-4d79-9958-813b51b9d104 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:28.641195 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93895fa-62ff-4d79-9958-813b51b9d104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:28.641602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93895fa-62ff-4d79-9958-813b51b9d104 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:28.642442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93895fa-62ff-4d79-9958-813b51b9d104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:28.645395 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93895fa-62ff-4d79-9958-813b51b9d104 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:28.667528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1284/2552] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:28.672592 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93895fa-62ff-4d79-9958-813b51b9d104 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:28.683709 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93895fa-62ff-4d79-9958-813b51b9d104 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:28.689663 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a5f81-2393-4401-9a55-b912ce4d17e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:28.690413 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a5f81-2393-4401-9a55-b912ce4d17e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:28.690782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a5f81-2393-4401-9a55-b912ce4d17e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:28.691135 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a5f81-2393-4401-9a55-b912ce4d17e1 admin admin] PATH_INFO: `/v3/projects/62023f115a114f98b2ffe693f1d37e54/users/e6ca4d0514694e0f85e955ff89283b54/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:28.712624 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a5f81-2393-4401-9a55-b912ce4d17e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62023f115a114f98b2ffe693f1d37e54, user_id=e6ca4d0514694e0f85e955ff89283b54, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:28.715753 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a5f81-2393-4401-9a55-b912ce4d17e1 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:28.800764 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1285/2553] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:28 2022] PUT /identity/v3/projects/62023f115a114f98b2ffe693f1d37e54/users/e6ca4d0514694e0f85e955ff89283b54/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:28.805000 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a5f81-2393-4401-9a55-b912ce4d17e1 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:28.818710 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-94dba65f-e38a-4fb5-9242-f8ce85e621f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2de56c261804bcb8b740ab5bacfd555', 'request_id': 'req-94dba65f-e38a-4fb5-9242-f8ce85e621f9'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:28.825245 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-94dba65f-e38a-4fb5-9242-f8ce85e621f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2de56c261804bcb8b740ab5bacfd555', 'request_id': 'req-94dba65f-e38a-4fb5-9242-f8ce85e621f9'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:28.833572 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-94dba65f-e38a-4fb5-9242-f8ce85e621f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2de56c261804bcb8b740ab5bacfd555', 'request_id': 'req-94dba65f-e38a-4fb5-9242-f8ce85e621f9'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:28.912962 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-94dba65f-e38a-4fb5-9242-f8ce85e621f9 admin admin] The token cache is being invalidate because project f2de56c261804bcb8b740ab5bacfd555 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:28.913643 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-94dba65f-e38a-4fb5-9242-f8ce85e621f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:28.914605 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-94dba65f-e38a-4fb5-9242-f8ce85e621f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:28.917663 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1269/2554] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:28 2022] DELETE /identity/v3/projects/f2de56c261804bcb8b740ab5bacfd555 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:28.922798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dba65f-e38a-4fb5-9242-f8ce85e621f9 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:28.930854 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-94dba65f-e38a-4fb5-9242-f8ce85e621f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:28.937394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a5f81-2393-4401-9a55-b912ce4d17e1 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:28.955900 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:29.012717 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:29.046720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dba65f-e38a-4fb5-9242-f8ce85e621f9 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:29.053915 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e675f-bff8-4565-b7bf-55d68c8e2a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:29.054872 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e675f-bff8-4565-b7bf-55d68c8e2a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:29.055212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e675f-bff8-4565-b7bf-55d68c8e2a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:29.058026 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e675f-bff8-4565-b7bf-55d68c8e2a9f admin admin] PATH_INFO: `/v3/users/5b7d845892b844ae81dca93940636ec7` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:29.058026 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e675f-bff8-4565-b7bf-55d68c8e2a9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b7d845892b844ae81dca93940636ec7)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:29.059251 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e675f-bff8-4565-b7bf-55d68c8e2a9f admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:29.070132 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b08ad-b246-420a-83d9-a3722f9a146c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:29.071040 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b08ad-b246-420a-83d9-a3722f9a146c admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:29.071529 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b08ad-b246-420a-83d9-a3722f9a146c admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:29.072111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b08ad-b246-420a-83d9-a3722f9a146c admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:29.073052 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b08ad-b246-420a-83d9-a3722f9a146c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:29.075222 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b08ad-b246-420a-83d9-a3722f9a146c admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:29.110750 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1286/2555] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:29.112044 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b08ad-b246-420a-83d9-a3722f9a146c admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:29.114851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-4b2b08ad-b246-420a-83d9-a3722f9a146c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:29.144638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b08ad-b246-420a-83d9-a3722f9a146c admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:29.162372 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a5ea9-1e7d-4c08-b315-de1c43fc63dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:29.164175 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a5ea9-1e7d-4c08-b315-de1c43fc63dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:29.164990 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a5ea9-1e7d-4c08-b315-de1c43fc63dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:29.165293 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a5ea9-1e7d-4c08-b315-de1c43fc63dc admin admin] PATH_INFO: `/v3/projects/62023f115a114f98b2ffe693f1d37e54/users/e6ca4d0514694e0f85e955ff89283b54/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:29.168427 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a5ea9-1e7d-4c08-b315-de1c43fc63dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=62023f115a114f98b2ffe693f1d37e54, user_id=e6ca4d0514694e0f85e955ff89283b54, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:29.177295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a5ea9-1e7d-4c08-b315-de1c43fc63dc admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:29.217884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1287/2556] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:29 2022] PUT /identity/v3/projects/62023f115a114f98b2ffe693f1d37e54/users/e6ca4d0514694e0f85e955ff89283b54/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:29.220431 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74425086-4d0b-45ff-875f-ad68e086ef0c None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:29.220853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74425086-4d0b-45ff-875f-ad68e086ef0c None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:29.221136 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74425086-4d0b-45ff-875f-ad68e086ef0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:29.293173 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-74425086-4d0b-45ff-875f-ad68e086ef0c None None] MFA Rules not processed for user `e6ca4d0514694e0f85e955ff89283b54`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:29.298602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-74425086-4d0b-45ff-875f-ad68e086ef0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:29.338881 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-3c7e675f-bff8-4565-b7bf-55d68c8e2a9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b7d845892b844ae81dca93940636ec7'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:29.431747 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-3c7e675f-bff8-4565-b7bf-55d68c8e2a9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b7d845892b844ae81dca93940636ec7'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:29.432379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-3c7e675f-bff8-4565-b7bf-55d68c8e2a9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b7d845892b844ae81dca93940636ec7'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:29.454762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-3c7e675f-bff8-4565-b7bf-55d68c8e2a9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b7d845892b844ae81dca93940636ec7'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:29.456435 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-3c7e675f-bff8-4565-b7bf-55d68c8e2a9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b7d845892b844ae81dca93940636ec7'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:29.463820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1288/2557] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:29 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:29.484417 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74425086-4d0b-45ff-875f-ad68e086ef0c None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:29.492781 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-74425086-4d0b-45ff-875f-ad68e086ef0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:29.514647 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1270/2558] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:28 2022] DELETE /identity/v3/users/5b7d845892b844ae81dca93940636ec7 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:29.646053 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74425086-4d0b-45ff-875f-ad68e086ef0c None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:29.744070 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b82f7-e6c3-4675-8520-8eb2c4ccc06a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:29.745523 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b82f7-e6c3-4675-8520-8eb2c4ccc06a service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:29.746213 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b82f7-e6c3-4675-8520-8eb2c4ccc06a service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:29.746938 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b82f7-e6c3-4675-8520-8eb2c4ccc06a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:29.838440 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:29.956582 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b82f7-e6c3-4675-8520-8eb2c4ccc06a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:29.959814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b82f7-e6c3-4675-8520-8eb2c4ccc06a service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:29.985016 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1289/2559] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:29 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:30.150667 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e675f-bff8-4565-b7bf-55d68c8e2a9f admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:30.154209 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-3c7e675f-bff8-4565-b7bf-55d68c8e2a9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:30.357576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b82f7-e6c3-4675-8520-8eb2c4ccc06a service cinder] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:30.359149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-0d4b82f7-e6c3-4675-8520-8eb2c4ccc06a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:30.374381 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e675f-bff8-4565-b7bf-55d68c8e2a9f admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:30.381478 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae384896-d350-477c-992f-f25a7e3feb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:30.382227 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae384896-d350-477c-992f-f25a7e3feb97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:30.382615 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae384896-d350-477c-992f-f25a7e3feb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:30.382945 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae384896-d350-477c-992f-f25a7e3feb97 admin admin] PATH_INFO: `/v3/projects/ffcbb561522a4367af3bde8aabb87c64` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:30.385257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae384896-d350-477c-992f-f25a7e3feb97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffcbb561522a4367af3bde8aabb87c64)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:30.386953 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b82f7-e6c3-4675-8520-8eb2c4ccc06a service cinder] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:30.387447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae384896-d350-477c-992f-f25a7e3feb97 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:30.393887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8509282-cc69-47cf-ae81-6d0039274c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:30.397539 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8509282-cc69-47cf-ae81-6d0039274c67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:30.397539 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8509282-cc69-47cf-ae81-6d0039274c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:30.397539 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8509282-cc69-47cf-ae81-6d0039274c67 admin admin] PATH_INFO: `/v3/users/59c7b8d09b5d4beaad0ad6430a80eda3` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:30.397539 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8509282-cc69-47cf-ae81-6d0039274c67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59c7b8d09b5d4beaad0ad6430a80eda3)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:30.403570 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8509282-cc69-47cf-ae81-6d0039274c67 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:30.542878 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:30.567682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-ae384896-d350-477c-992f-f25a7e3feb97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffcbb561522a4367af3bde8aabb87c64', 'request_id': 'req-ae384896-d350-477c-992f-f25a7e3feb97'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.581925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-ae384896-d350-477c-992f-f25a7e3feb97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffcbb561522a4367af3bde8aabb87c64', 'request_id': 'req-ae384896-d350-477c-992f-f25a7e3feb97'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.604200 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-ae384896-d350-477c-992f-f25a7e3feb97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffcbb561522a4367af3bde8aabb87c64', 'request_id': 'req-ae384896-d350-477c-992f-f25a7e3feb97'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.664449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-c8509282-cc69-47cf-ae81-6d0039274c67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59c7b8d09b5d4beaad0ad6430a80eda3'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.722902 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-ae384896-d350-477c-992f-f25a7e3feb97 admin admin] The token cache is being invalidate because project ffcbb561522a4367af3bde8aabb87c64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:30.724971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-ae384896-d350-477c-992f-f25a7e3feb97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.724971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-ae384896-d350-477c-992f-f25a7e3feb97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.725733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1271/2560] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:30 2022] DELETE /identity/v3/projects/ffcbb561522a4367af3bde8aabb87c64 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:30.740646 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae384896-d350-477c-992f-f25a7e3feb97 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:30.745287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ae384896-d350-477c-992f-f25a7e3feb97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:30.825096 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-c8509282-cc69-47cf-ae81-6d0039274c67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59c7b8d09b5d4beaad0ad6430a80eda3'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.826619 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-c8509282-cc69-47cf-ae81-6d0039274c67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59c7b8d09b5d4beaad0ad6430a80eda3'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.844479 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-c8509282-cc69-47cf-ae81-6d0039274c67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59c7b8d09b5d4beaad0ad6430a80eda3'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.850435 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae384896-d350-477c-992f-f25a7e3feb97 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:30.869091 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faded855-45bc-4384-9809-b4c5596c50e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:30.869091 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faded855-45bc-4384-9809-b4c5596c50e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:30.869091 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faded855-45bc-4384-9809-b4c5596c50e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:30.869091 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faded855-45bc-4384-9809-b4c5596c50e3 admin admin] PATH_INFO: `/v3/users/7be6501df0c84b1c9dd0e87dd67e21f3` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:30.870437 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faded855-45bc-4384-9809-b4c5596c50e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7be6501df0c84b1c9dd0e87dd67e21f3)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:30.873623 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-c8509282-cc69-47cf-ae81-6d0039274c67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59c7b8d09b5d4beaad0ad6430a80eda3'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.877191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faded855-45bc-4384-9809-b4c5596c50e3 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:30.889482 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1290/2561] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:30 2022] DELETE /identity/v3/users/59c7b8d09b5d4beaad0ad6430a80eda3 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:30.894148 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54990fae-1d2a-4e84-ab4f-52fdc9071492 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:30.894495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54990fae-1d2a-4e84-ab4f-52fdc9071492 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:30.894839 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54990fae-1d2a-4e84-ab4f-52fdc9071492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:30.920095 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-54990fae-1d2a-4e84-ab4f-52fdc9071492 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:30.921835 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-54990fae-1d2a-4e84-ab4f-52fdc9071492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:31.005841 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1291/2562] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:31.019651 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54990fae-1d2a-4e84-ab4f-52fdc9071492 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:31.024241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-54990fae-1d2a-4e84-ab4f-52fdc9071492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:31.091072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:31.112933 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-faded855-45bc-4384-9809-b4c5596c50e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7be6501df0c84b1c9dd0e87dd67e21f3'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:31.118381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54990fae-1d2a-4e84-ab4f-52fdc9071492 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:31.123860 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa042e-5d9e-458c-96fb-e75230b4809a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:31.123860 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa042e-5d9e-458c-96fb-e75230b4809a service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:31.123860 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa042e-5d9e-458c-96fb-e75230b4809a service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:31.124494 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa042e-5d9e-458c-96fb-e75230b4809a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:31.187323 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa042e-5d9e-458c-96fb-e75230b4809a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:31.195631 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa042e-5d9e-458c-96fb-e75230b4809a service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:31.224683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-faded855-45bc-4384-9809-b4c5596c50e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7be6501df0c84b1c9dd0e87dd67e21f3'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:31.225556 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-faded855-45bc-4384-9809-b4c5596c50e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7be6501df0c84b1c9dd0e87dd67e21f3'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:31.243246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1292/2563] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:31.247181 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-faded855-45bc-4384-9809-b4c5596c50e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7be6501df0c84b1c9dd0e87dd67e21f3'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:31.249091 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-faded855-45bc-4384-9809-b4c5596c50e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7be6501df0c84b1c9dd0e87dd67e21f3'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:31.298122 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1272/2564] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:30 2022] DELETE /identity/v3/users/7be6501df0c84b1c9dd0e87dd67e21f3 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:32.046975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa042e-5d9e-458c-96fb-e75230b4809a service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:32.048839 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-9aaa042e-5d9e-458c-96fb-e75230b4809a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:32.086954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa042e-5d9e-458c-96fb-e75230b4809a service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:32.118610 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faded855-45bc-4384-9809-b4c5596c50e3 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:32.120611 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-faded855-45bc-4384-9809-b4c5596c50e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:32.243907 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b1651-5a5d-48bf-9dbd-d4d7e2adede7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:32.243907 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b1651-5a5d-48bf-9dbd-d4d7e2adede7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:32.244274 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b1651-5a5d-48bf-9dbd-d4d7e2adede7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:32.244475 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b1651-5a5d-48bf-9dbd-d4d7e2adede7 admin admin] PATH_INFO: `/v3/projects/d336332be1b54b11983c68924779945e` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:32.245722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b1651-5a5d-48bf-9dbd-d4d7e2adede7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d336332be1b54b11983c68924779945e)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:32.249851 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b1651-5a5d-48bf-9dbd-d4d7e2adede7 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:32.288181 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:32.324454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faded855-45bc-4384-9809-b4c5596c50e3 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:32.330945 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c5d19-fe26-44c1-8d73-7ec10280168e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:32.331630 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c5d19-fe26-44c1-8d73-7ec10280168e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:32.331911 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c5d19-fe26-44c1-8d73-7ec10280168e admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:32.332183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c5d19-fe26-44c1-8d73-7ec10280168e admin admin] PATH_INFO: `/v3/projects/0abf2bede458497789f8e7c50014873f` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:32.333454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c5d19-fe26-44c1-8d73-7ec10280168e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0abf2bede458497789f8e7c50014873f)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:32.335471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c5d19-fe26-44c1-8d73-7ec10280168e admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:32.483588 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0a1b1651-5a5d-48bf-9dbd-d4d7e2adede7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd336332be1b54b11983c68924779945e', 'request_id': 'req-0a1b1651-5a5d-48bf-9dbd-d4d7e2adede7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:32.504282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-510c5d19-fe26-44c1-8d73-7ec10280168e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0abf2bede458497789f8e7c50014873f', 'request_id': 'req-510c5d19-fe26-44c1-8d73-7ec10280168e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:32.511185 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-510c5d19-fe26-44c1-8d73-7ec10280168e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0abf2bede458497789f8e7c50014873f', 'request_id': 'req-510c5d19-fe26-44c1-8d73-7ec10280168e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:32.526044 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-510c5d19-fe26-44c1-8d73-7ec10280168e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0abf2bede458497789f8e7c50014873f', 'request_id': 'req-510c5d19-fe26-44c1-8d73-7ec10280168e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:32.598443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0a1b1651-5a5d-48bf-9dbd-d4d7e2adede7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd336332be1b54b11983c68924779945e', 'request_id': 'req-0a1b1651-5a5d-48bf-9dbd-d4d7e2adede7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:32.621426 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0a1b1651-5a5d-48bf-9dbd-d4d7e2adede7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd336332be1b54b11983c68924779945e', 'request_id': 'req-0a1b1651-5a5d-48bf-9dbd-d4d7e2adede7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:32.646630 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0a1b1651-5a5d-48bf-9dbd-d4d7e2adede7 admin admin] The token cache is being invalidate because project d336332be1b54b11983c68924779945e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:32.647006 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0a1b1651-5a5d-48bf-9dbd-d4d7e2adede7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:32.648103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0a1b1651-5a5d-48bf-9dbd-d4d7e2adede7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:32.649698 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1293/2565] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:32 2022] DELETE /identity/v3/projects/d336332be1b54b11983c68924779945e => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:32.655448 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b1651-5a5d-48bf-9dbd-d4d7e2adede7 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:32.657280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-0a1b1651-5a5d-48bf-9dbd-d4d7e2adede7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:32.666952 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-510c5d19-fe26-44c1-8d73-7ec10280168e admin admin] The token cache is being invalidate because project 0abf2bede458497789f8e7c50014873f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:32.667569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-510c5d19-fe26-44c1-8d73-7ec10280168e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:32.669052 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-510c5d19-fe26-44c1-8d73-7ec10280168e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:32.672762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1273/2566] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:32 2022] DELETE /identity/v3/projects/0abf2bede458497789f8e7c50014873f => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:32.680636 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f68083-44c7-4e6d-8629-c783777c7883 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:32.681247 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f68083-44c7-4e6d-8629-c783777c7883 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:32.681669 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f68083-44c7-4e6d-8629-c783777c7883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:32.724463 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-79f68083-44c7-4e6d-8629-c783777c7883 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:32.726118 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-79f68083-44c7-4e6d-8629-c783777c7883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:32.798381 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b1651-5a5d-48bf-9dbd-d4d7e2adede7 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:32.812537 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:32.862917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1274/2567] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:32.864416 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa064d9-e853-4513-aa19-846fa1ef0775 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:32.864873 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa064d9-e853-4513-aa19-846fa1ef0775 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:32.865020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa064d9-e853-4513-aa19-846fa1ef0775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:32.898620 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-2aa064d9-e853-4513-aa19-846fa1ef0775 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:32.898620 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-2aa064d9-e853-4513-aa19-846fa1ef0775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:32.899572 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1275/2568] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:32.904954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa064d9-e853-4513-aa19-846fa1ef0775 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:32.922802 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa064d9-e853-4513-aa19-846fa1ef0775 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:32.927231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eb62b5-63eb-4abd-b4d5-1e4f4b4d2f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:32.928245 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb62b5-63eb-4abd-b4d5-1e4f4b4d2f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:32.928600 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb62b5-63eb-4abd-b4d5-1e4f4b4d2f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:32.928779 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb62b5-63eb-4abd-b4d5-1e4f4b4d2f48 admin admin] PATH_INFO: `/v3/users/e6ca4d0514694e0f85e955ff89283b54` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:32.930446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eb62b5-63eb-4abd-b4d5-1e4f4b4d2f48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6ca4d0514694e0f85e955ff89283b54)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:32.932878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eb62b5-63eb-4abd-b4d5-1e4f4b4d2f48 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:32.938654 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fe7a1-43b2-4253-bd0e-27b70972cf08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:32.939392 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fe7a1-43b2-4253-bd0e-27b70972cf08 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:32.939783 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fe7a1-43b2-4253-bd0e-27b70972cf08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:32.940180 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fe7a1-43b2-4253-bd0e-27b70972cf08 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:32.940992 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47fe7a1-43b2-4253-bd0e-27b70972cf08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:32.941426 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47fe7a1-43b2-4253-bd0e-27b70972cf08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:32.943367 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47fe7a1-43b2-4253-bd0e-27b70972cf08 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:32.966916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1276/2569] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:32.976498 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fe7a1-43b2-4253-bd0e-27b70972cf08 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:32.991890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fe7a1-43b2-4253-bd0e-27b70972cf08 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:32.997279 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76914d-48fc-4a29-a32e-fd34c0ec094d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:32.998462 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76914d-48fc-4a29-a32e-fd34c0ec094d admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:32.998772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76914d-48fc-4a29-a32e-fd34c0ec094d admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:32.999114 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76914d-48fc-4a29-a32e-fd34c0ec094d admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:32.999755 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76914d-48fc-4a29-a32e-fd34c0ec094d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:33.001737 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76914d-48fc-4a29-a32e-fd34c0ec094d admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:33.147602 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1277/2570] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:32 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:33.152370 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76914d-48fc-4a29-a32e-fd34c0ec094d admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:33.269106 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76914d-48fc-4a29-a32e-fd34c0ec094d admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:33.275518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ed99a-80b8-4569-9bcc-6d8fbb8ec364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:33.276465 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ed99a-80b8-4569-9bcc-6d8fbb8ec364 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:33.277039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ed99a-80b8-4569-9bcc-6d8fbb8ec364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:33.277474 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ed99a-80b8-4569-9bcc-6d8fbb8ec364 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:33.278434 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ed99a-80b8-4569-9bcc-6d8fbb8ec364 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:33.282402 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ed99a-80b8-4569-9bcc-6d8fbb8ec364 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:33.483256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-85eb62b5-63eb-4abd-b4d5-1e4f4b4d2f48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6ca4d0514694e0f85e955ff89283b54'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:33.532464 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1278/2571] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:33 2022] POST /identity/v3/users => generated 292 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:33.537927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ed99a-80b8-4569-9bcc-6d8fbb8ec364 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:33.548773 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ed99a-80b8-4569-9bcc-6d8fbb8ec364 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:33.553945 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664d74d-22bd-4056-97d2-9e62138409bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:33.555003 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664d74d-22bd-4056-97d2-9e62138409bd admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:33.555512 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664d74d-22bd-4056-97d2-9e62138409bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:33.555949 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664d74d-22bd-4056-97d2-9e62138409bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:33.556791 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c664d74d-22bd-4056-97d2-9e62138409bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:33.559132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c664d74d-22bd-4056-97d2-9e62138409bd admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:33.563741 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-85eb62b5-63eb-4abd-b4d5-1e4f4b4d2f48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6ca4d0514694e0f85e955ff89283b54'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:33.567134 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-85eb62b5-63eb-4abd-b4d5-1e4f4b4d2f48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6ca4d0514694e0f85e955ff89283b54'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:33.567278 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:33.574521 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-85eb62b5-63eb-4abd-b4d5-1e4f4b4d2f48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6ca4d0514694e0f85e955ff89283b54'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:33.589642 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1279/2572] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:33.595161 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-85eb62b5-63eb-4abd-b4d5-1e4f4b4d2f48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6ca4d0514694e0f85e955ff89283b54'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:33.595923 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664d74d-22bd-4056-97d2-9e62138409bd admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:33.599382 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-c664d74d-22bd-4056-97d2-9e62138409bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:33.607918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1294/2573] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:32 2022] DELETE /identity/v3/users/e6ca4d0514694e0f85e955ff89283b54 => generated 0 bytes in 950 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:33.617091 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664d74d-22bd-4056-97d2-9e62138409bd admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:33.693180 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828642e-b8e3-4310-8040-3388a5fb1020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:33.693703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828642e-b8e3-4310-8040-3388a5fb1020 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:33.693986 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828642e-b8e3-4310-8040-3388a5fb1020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:33.694293 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828642e-b8e3-4310-8040-3388a5fb1020 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:33.695021 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d828642e-b8e3-4310-8040-3388a5fb1020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:33.696917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d828642e-b8e3-4310-8040-3388a5fb1020 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:33.710950 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1280/2574] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:33.715134 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eb62b5-63eb-4abd-b4d5-1e4f4b4d2f48 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:33.728201 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eb62b5-63eb-4abd-b4d5-1e4f4b4d2f48 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:33.734294 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316b679-4631-4a76-810b-830f05a452af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:33.735035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316b679-4631-4a76-810b-830f05a452af admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:33.735453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316b679-4631-4a76-810b-830f05a452af admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:33.735874 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316b679-4631-4a76-810b-830f05a452af admin admin] PATH_INFO: `/v3/projects/ea76fc5a846b4a9d98a862861fdbba4b/users/c35b89ac8c8b4e5aa49c004fd3bff45b/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:33.754140 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316b679-4631-4a76-810b-830f05a452af admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea76fc5a846b4a9d98a862861fdbba4b, user_id=c35b89ac8c8b4e5aa49c004fd3bff45b, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:33.756859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316b679-4631-4a76-810b-830f05a452af admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:33.851206 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1295/2575] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:33 2022] PUT /identity/v3/projects/ea76fc5a846b4a9d98a862861fdbba4b/users/c35b89ac8c8b4e5aa49c004fd3bff45b/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:33.853077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d31b0-0fde-46e2-8977-88d023087d49 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:33.853638 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d31b0-0fde-46e2-8977-88d023087d49 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:33.854123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d31b0-0fde-46e2-8977-88d023087d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:33.894124 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-b42d31b0-0fde-46e2-8977-88d023087d49 None None] MFA Rules not processed for user `c35b89ac8c8b4e5aa49c004fd3bff45b`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:33.898561 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-b42d31b0-0fde-46e2-8977-88d023087d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:34.037609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1281/2576] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:34.043624 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316b679-4631-4a76-810b-830f05a452af admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:34.193580 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316b679-4631-4a76-810b-830f05a452af admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:34.207754 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5b90b-cfee-4fcf-96ce-91466aa1f2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:34.208493 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5b90b-cfee-4fcf-96ce-91466aa1f2ac admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:34.208898 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5b90b-cfee-4fcf-96ce-91466aa1f2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:34.209267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5b90b-cfee-4fcf-96ce-91466aa1f2ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:34.210019 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5b90b-cfee-4fcf-96ce-91466aa1f2ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:34.212191 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5b90b-cfee-4fcf-96ce-91466aa1f2ac admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:34.233219 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:34.342096 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1296/2577] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:34 2022] POST /identity/v3/projects => generated 385 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:34.345988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d31b0-0fde-46e2-8977-88d023087d49 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:34.346951 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-b42d31b0-0fde-46e2-8977-88d023087d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:34.348830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5b90b-cfee-4fcf-96ce-91466aa1f2ac admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:34.372323 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d31b0-0fde-46e2-8977-88d023087d49 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:34.441456 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:34.452346 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5b90b-cfee-4fcf-96ce-91466aa1f2ac admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:34.457750 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a97133-444d-45bb-9c1e-a794dc694d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:34.458472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a97133-444d-45bb-9c1e-a794dc694d1f admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:34.458748 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a97133-444d-45bb-9c1e-a794dc694d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:34.459052 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a97133-444d-45bb-9c1e-a794dc694d1f admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:34.459665 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a97133-444d-45bb-9c1e-a794dc694d1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:34.463019 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a97133-444d-45bb-9c1e-a794dc694d1f admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:34.480314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f01f21-5035-4529-a10d-640dff19e578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:34.480957 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f01f21-5035-4529-a10d-640dff19e578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:34.485778 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f01f21-5035-4529-a10d-640dff19e578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:34.485778 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f01f21-5035-4529-a10d-640dff19e578 admin admin] PATH_INFO: `/v3/projects/62023f115a114f98b2ffe693f1d37e54` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:34.485778 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f01f21-5035-4529-a10d-640dff19e578 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62023f115a114f98b2ffe693f1d37e54)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:34.485778 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f01f21-5035-4529-a10d-640dff19e578 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:34.569426 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1297/2578] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:34 2022] POST /identity/v3/users => generated 299 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:34.573377 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a97133-444d-45bb-9c1e-a794dc694d1f admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:34.586333 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a97133-444d-45bb-9c1e-a794dc694d1f admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:34.588889 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca0975-a356-4718-a6d6-ee5b337e84f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:34.589521 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca0975-a356-4718-a6d6-ee5b337e84f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:34.589793 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca0975-a356-4718-a6d6-ee5b337e84f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:34.590100 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca0975-a356-4718-a6d6-ee5b337e84f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:34.590960 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca0975-a356-4718-a6d6-ee5b337e84f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:34.593377 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca0975-a356-4718-a6d6-ee5b337e84f0 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:34.604544 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1298/2579] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:34.609566 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca0975-a356-4718-a6d6-ee5b337e84f0 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:34.619413 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca0975-a356-4718-a6d6-ee5b337e84f0 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:34.631682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-44f01f21-5035-4529-a10d-640dff19e578 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62023f115a114f98b2ffe693f1d37e54', 'request_id': 'req-44f01f21-5035-4529-a10d-640dff19e578'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:34.637146 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-44f01f21-5035-4529-a10d-640dff19e578 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62023f115a114f98b2ffe693f1d37e54', 'request_id': 'req-44f01f21-5035-4529-a10d-640dff19e578'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:34.644304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-44f01f21-5035-4529-a10d-640dff19e578 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62023f115a114f98b2ffe693f1d37e54', 'request_id': 'req-44f01f21-5035-4529-a10d-640dff19e578'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:34.681108 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4f20b-f9d5-4f97-9bd9-f72c57c4aca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:34.681809 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4f20b-f9d5-4f97-9bd9-f72c57c4aca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:34.682092 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4f20b-f9d5-4f97-9bd9-f72c57c4aca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:34.682356 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4f20b-f9d5-4f97-9bd9-f72c57c4aca1 admin admin] PATH_INFO: `/v3/projects/f06a92e9e9da47dea03379108050202c/users/154afa43db3349749366ebb7394c0e6c/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:34.698652 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4f20b-f9d5-4f97-9bd9-f72c57c4aca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f06a92e9e9da47dea03379108050202c, user_id=154afa43db3349749366ebb7394c0e6c, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:34.701820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4f20b-f9d5-4f97-9bd9-f72c57c4aca1 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:34.718223 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-44f01f21-5035-4529-a10d-640dff19e578 admin admin] The token cache is being invalidate because project 62023f115a114f98b2ffe693f1d37e54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:34.719838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-44f01f21-5035-4529-a10d-640dff19e578 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:34.719838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-44f01f21-5035-4529-a10d-640dff19e578 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:34.720832 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1282/2580] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:34 2022] DELETE /identity/v3/projects/62023f115a114f98b2ffe693f1d37e54 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:34.726092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c489a-d3e1-4339-a933-74b53fa2c877 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:34.726433 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c489a-d3e1-4339-a933-74b53fa2c877 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:34.726766 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c489a-d3e1-4339-a933-74b53fa2c877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:34.749729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-430c489a-d3e1-4339-a933-74b53fa2c877 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:34.751116 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-430c489a-d3e1-4339-a933-74b53fa2c877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:34.759093 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1283/2581] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:34.767919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78e4a6-f1a1-4ef2-8992-15f92153260e None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:34.768199 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78e4a6-f1a1-4ef2-8992-15f92153260e None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:34.768468 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78e4a6-f1a1-4ef2-8992-15f92153260e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:34.790891 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-ab78e4a6-f1a1-4ef2-8992-15f92153260e None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:34.792092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ab78e4a6-f1a1-4ef2-8992-15f92153260e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:34.802350 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1284/2582] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:34.806536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1299/2583] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:34 2022] PUT /identity/v3/projects/f06a92e9e9da47dea03379108050202c/users/154afa43db3349749366ebb7394c0e6c/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:34.807800 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78e4a6-f1a1-4ef2-8992-15f92153260e None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:34.811531 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4f20b-f9d5-4f97-9bd9-f72c57c4aca1 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:34.813334 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-11a4f20b-f9d5-4f97-9bd9-f72c57c4aca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:34.822720 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78e4a6-f1a1-4ef2-8992-15f92153260e None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:34.882693 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4f20b-f9d5-4f97-9bd9-f72c57c4aca1 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:34.887740 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa47e3-3128-4f52-bb36-f17e6dc8172c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:34.888801 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa47e3-3128-4f52-bb36-f17e6dc8172c admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:34.889149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa47e3-3128-4f52-bb36-f17e6dc8172c admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:34.889421 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa47e3-3128-4f52-bb36-f17e6dc8172c admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:34.890063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa47e3-3128-4f52-bb36-f17e6dc8172c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:34.890356 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c967a-6b37-49c7-a685-4ad132e5bd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:34.891124 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c967a-6b37-49c7-a685-4ad132e5bd9f admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:34.891498 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c967a-6b37-49c7-a685-4ad132e5bd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:34.891856 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa47e3-3128-4f52-bb36-f17e6dc8172c admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:34.891964 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c967a-6b37-49c7-a685-4ad132e5bd9f admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:34.892775 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c967a-6b37-49c7-a685-4ad132e5bd9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:34.893137 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c967a-6b37-49c7-a685-4ad132e5bd9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:34.894895 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c967a-6b37-49c7-a685-4ad132e5bd9f admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:34.902420 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1300/2584] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:34.907190 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa47e3-3128-4f52-bb36-f17e6dc8172c admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:34.910751 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1285/2585] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:34.914427 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c967a-6b37-49c7-a685-4ad132e5bd9f admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:34.917722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa47e3-3128-4f52-bb36-f17e6dc8172c admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:34.923468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a657b81-5f48-412a-8d51-e4a8dc207349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:34.924118 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a657b81-5f48-412a-8d51-e4a8dc207349 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:34.924564 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a657b81-5f48-412a-8d51-e4a8dc207349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:34.924899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a657b81-5f48-412a-8d51-e4a8dc207349 admin admin] PATH_INFO: `/v3/projects/f06a92e9e9da47dea03379108050202c/users/154afa43db3349749366ebb7394c0e6c/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:34.925592 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c967a-6b37-49c7-a685-4ad132e5bd9f admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:34.927168 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a657b81-5f48-412a-8d51-e4a8dc207349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f06a92e9e9da47dea03379108050202c, user_id=154afa43db3349749366ebb7394c0e6c, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:34.929683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a657b81-5f48-412a-8d51-e4a8dc207349 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:34.930770 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9a2fe-bf61-45da-a6fd-5c6e87f6c42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:34.931384 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9a2fe-bf61-45da-a6fd-5c6e87f6c42d admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:34.931868 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9a2fe-bf61-45da-a6fd-5c6e87f6c42d admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:34.932169 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9a2fe-bf61-45da-a6fd-5c6e87f6c42d admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:34.932794 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9a2fe-bf61-45da-a6fd-5c6e87f6c42d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:34.934868 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9a2fe-bf61-45da-a6fd-5c6e87f6c42d admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:34.942171 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1301/2586] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:34 2022] PUT /identity/v3/projects/f06a92e9e9da47dea03379108050202c/users/154afa43db3349749366ebb7394c0e6c/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:34.958407 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060deb7-a792-43b8-8992-a4a481b045c2 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:34.958730 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060deb7-a792-43b8-8992-a4a481b045c2 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:34.959005 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060deb7-a792-43b8-8992-a4a481b045c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:34.969914 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:34.989312 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-f060deb7-a792-43b8-8992-a4a481b045c2 None None] MFA Rules not processed for user `154afa43db3349749366ebb7394c0e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:34.990595 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f060deb7-a792-43b8-8992-a4a481b045c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:35.057274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1286/2587] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:34 2022] POST /identity/v3/projects => generated 405 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:35.064386 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9a2fe-bf61-45da-a6fd-5c6e87f6c42d admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:35.078599 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1302/2588] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:34 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:35.089968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f060deb7-a792-43b8-8992-a4a481b045c2 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:35.092376 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f060deb7-a792-43b8-8992-a4a481b045c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:35.101479 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:35.140625 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9a2fe-bf61-45da-a6fd-5c6e87f6c42d admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:35.146094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5498998-5d84-4de8-98f7-66350c8ff7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:35.146728 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5498998-5d84-4de8-98f7-66350c8ff7e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:35.147009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5498998-5d84-4de8-98f7-66350c8ff7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:35.147339 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5498998-5d84-4de8-98f7-66350c8ff7e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:35.147959 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5498998-5d84-4de8-98f7-66350c8ff7e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:35.149934 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5498998-5d84-4de8-98f7-66350c8ff7e6 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:35.159105 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f060deb7-a792-43b8-8992-a4a481b045c2 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:35.212434 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45bd09a-60a1-43c0-bdaa-a60b3f1e1bea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:35.213058 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bd09a-60a1-43c0-bdaa-a60b3f1e1bea service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:35.213339 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bd09a-60a1-43c0-bdaa-a60b3f1e1bea service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:35.213611 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bd09a-60a1-43c0-bdaa-a60b3f1e1bea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:35.260421 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1287/2589] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:35 2022] POST /identity/v3/users => generated 303 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:35.267025 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5498998-5d84-4de8-98f7-66350c8ff7e6 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:35.278412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5498998-5d84-4de8-98f7-66350c8ff7e6 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:35.283477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056422e-d29b-4ab1-96ee-3aa9769d37df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:35.284091 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056422e-d29b-4ab1-96ee-3aa9769d37df admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:35.284376 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056422e-d29b-4ab1-96ee-3aa9769d37df admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:35.284644 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056422e-d29b-4ab1-96ee-3aa9769d37df admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:35.285290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056422e-d29b-4ab1-96ee-3aa9769d37df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:35.286936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056422e-d29b-4ab1-96ee-3aa9769d37df admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:35.290896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45bd09a-60a1-43c0-bdaa-a60b3f1e1bea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:35.292408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45bd09a-60a1-43c0-bdaa-a60b3f1e1bea service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:35.299211 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1288/2590] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:35.304141 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1303/2591] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:35 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:35.304870 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056422e-d29b-4ab1-96ee-3aa9769d37df admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:35.322894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056422e-d29b-4ab1-96ee-3aa9769d37df admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:35.331767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731c690-1d05-4da8-9df3-d0641f08cea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:35.332844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731c690-1d05-4da8-9df3-d0641f08cea0 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:35.333226 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731c690-1d05-4da8-9df3-d0641f08cea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:35.333544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731c690-1d05-4da8-9df3-d0641f08cea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:35.334596 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1731c690-1d05-4da8-9df3-d0641f08cea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:35.336363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1731c690-1d05-4da8-9df3-d0641f08cea0 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:35.350279 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1289/2592] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:35.353523 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45bd09a-60a1-43c0-bdaa-a60b3f1e1bea service cinder] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:35.365468 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45bd09a-60a1-43c0-bdaa-a60b3f1e1bea service cinder] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:35.370408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58abc5-fdc8-4c0e-a664-af612020631a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:35.371123 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58abc5-fdc8-4c0e-a664-af612020631a admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:35.371401 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58abc5-fdc8-4c0e-a664-af612020631a admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:35.371672 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58abc5-fdc8-4c0e-a664-af612020631a admin admin] PATH_INFO: `/v3/projects/3ac2d654c4ba4073861b517518a3b5d4/users/9963f7d305294acc841ad567899383cf/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:35.388968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58abc5-fdc8-4c0e-a664-af612020631a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ac2d654c4ba4073861b517518a3b5d4, user_id=9963f7d305294acc841ad567899383cf, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:35.391701 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58abc5-fdc8-4c0e-a664-af612020631a admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:35.432124 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:35.539373 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1304/2593] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:35 2022] PUT /identity/v3/projects/3ac2d654c4ba4073861b517518a3b5d4/users/9963f7d305294acc841ad567899383cf/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:35.545238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec02780-6e09-4d55-b289-6d4f69b310e3 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:35.545508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec02780-6e09-4d55-b289-6d4f69b310e3 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:35.545785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec02780-6e09-4d55-b289-6d4f69b310e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:35.565664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-1ec02780-6e09-4d55-b289-6d4f69b310e3 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:35.566981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-1ec02780-6e09-4d55-b289-6d4f69b310e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:35.635954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1290/2594] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:35.646873 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58abc5-fdc8-4c0e-a664-af612020631a admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:35.648842 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-3c58abc5-fdc8-4c0e-a664-af612020631a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:35.736938 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58abc5-fdc8-4c0e-a664-af612020631a admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:35.740849 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77a39f-4d98-4d9e-90d6-bbf817254df1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:35.741483 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77a39f-4d98-4d9e-90d6-bbf817254df1 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:35.741757 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77a39f-4d98-4d9e-90d6-bbf817254df1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:35.742024 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77a39f-4d98-4d9e-90d6-bbf817254df1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:35.754003 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77a39f-4d98-4d9e-90d6-bbf817254df1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:35.756151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77a39f-4d98-4d9e-90d6-bbf817254df1 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:35.768973 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1305/2595] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:36.913755 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec02780-6e09-4d55-b289-6d4f69b310e3 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:36.930775 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec02780-6e09-4d55-b289-6d4f69b310e3 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:36.940625 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe57f23-6b71-4ff7-b84e-381e6c5f2d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:36.940899 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe57f23-6b71-4ff7-b84e-381e6c5f2d49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:36.941314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe57f23-6b71-4ff7-b84e-381e6c5f2d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:36.941590 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe57f23-6b71-4ff7-b84e-381e6c5f2d49 admin admin] PATH_INFO: `/v3/users/c35b89ac8c8b4e5aa49c004fd3bff45b` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:36.943080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe57f23-6b71-4ff7-b84e-381e6c5f2d49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c35b89ac8c8b4e5aa49c004fd3bff45b)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:36.945286 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe57f23-6b71-4ff7-b84e-381e6c5f2d49 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:38.126209 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-6fe57f23-6b71-4ff7-b84e-381e6c5f2d49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c35b89ac8c8b4e5aa49c004fd3bff45b'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:38.214664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-6fe57f23-6b71-4ff7-b84e-381e6c5f2d49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c35b89ac8c8b4e5aa49c004fd3bff45b'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:38.215379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-6fe57f23-6b71-4ff7-b84e-381e6c5f2d49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c35b89ac8c8b4e5aa49c004fd3bff45b'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:38.227003 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-6fe57f23-6b71-4ff7-b84e-381e6c5f2d49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c35b89ac8c8b4e5aa49c004fd3bff45b'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:38.227977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-6fe57f23-6b71-4ff7-b84e-381e6c5f2d49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c35b89ac8c8b4e5aa49c004fd3bff45b'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:38.247975 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1291/2596] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:36 2022] DELETE /identity/v3/users/c35b89ac8c8b4e5aa49c004fd3bff45b => generated 0 bytes in 1335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:38.253356 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180de03-f33f-4b43-92d1-d87791693b17 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:38.253849 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180de03-f33f-4b43-92d1-d87791693b17 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:38.261975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180de03-f33f-4b43-92d1-d87791693b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:38.325302 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8f14a-2451-4622-8a73-3d530e139734 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:38.325776 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8f14a-2451-4622-8a73-3d530e139734 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:38.326256 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8f14a-2451-4622-8a73-3d530e139734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:38.355105 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:38.388662 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-0fe8f14a-2451-4622-8a73-3d530e139734 None None] MFA Rules not processed for user `9963f7d305294acc841ad567899383cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:38.390182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-0fe8f14a-2451-4622-8a73-3d530e139734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:38.410681 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-9180de03-f33f-4b43-92d1-d87791693b17 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:38.413122 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-9180de03-f33f-4b43-92d1-d87791693b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:38.478943 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1292/2597] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:38 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:38.481613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8f14a-2451-4622-8a73-3d530e139734 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:38.483117 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-0fe8f14a-2451-4622-8a73-3d530e139734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:38.509185 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1306/2598] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:38.519308 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180de03-f33f-4b43-92d1-d87791693b17 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:38.520778 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-9180de03-f33f-4b43-92d1-d87791693b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:38.555711 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8f14a-2451-4622-8a73-3d530e139734 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:38.560683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b0724-e6b3-4d4b-9e4d-910e4848d6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:38.561574 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b0724-e6b3-4d4b-9e4d-910e4848d6c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:38.561843 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b0724-e6b3-4d4b-9e4d-910e4848d6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:38.562104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b0724-e6b3-4d4b-9e4d-910e4848d6c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:38.564073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b0724-e6b3-4d4b-9e4d-910e4848d6c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:38.566145 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b0724-e6b3-4d4b-9e4d-910e4848d6c9 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:38.588010 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:38.591776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180de03-f33f-4b43-92d1-d87791693b17 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:38.596101 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a355f1-6e13-4201-a0ed-832433e8dbbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:38.598856 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a355f1-6e13-4201-a0ed-832433e8dbbf service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:38.598856 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a355f1-6e13-4201-a0ed-832433e8dbbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:38.598856 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a355f1-6e13-4201-a0ed-832433e8dbbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:38.606251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a355f1-6e13-4201-a0ed-832433e8dbbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:38.607706 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a355f1-6e13-4201-a0ed-832433e8dbbf service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:38.618341 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1307/2599] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:38.667810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1293/2600] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:38 2022] POST /identity/v3/projects => generated 405 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:38.671313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a355f1-6e13-4201-a0ed-832433e8dbbf service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:38.685472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a355f1-6e13-4201-a0ed-832433e8dbbf service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:38.761999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda282b-2583-4601-8927-c58ccf1692ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:38.762648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda282b-2583-4601-8927-c58ccf1692ad admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:38.762937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda282b-2583-4601-8927-c58ccf1692ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:38.763189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda282b-2583-4601-8927-c58ccf1692ad admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:38.763830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda282b-2583-4601-8927-c58ccf1692ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:38.766888 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda282b-2583-4601-8927-c58ccf1692ad admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:38.855918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1308/2601] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:38 2022] POST /identity/v3/users => generated 309 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:38.860753 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b0724-e6b3-4d4b-9e4d-910e4848d6c9 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:38.884524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b0724-e6b3-4d4b-9e4d-910e4848d6c9 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:38.896032 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c69a06-85c9-476d-b638-67bc0f12e966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:38.896662 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c69a06-85c9-476d-b638-67bc0f12e966 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:38.896943 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c69a06-85c9-476d-b638-67bc0f12e966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:38.897212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c69a06-85c9-476d-b638-67bc0f12e966 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:38.897865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c69a06-85c9-476d-b638-67bc0f12e966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:38.899650 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c69a06-85c9-476d-b638-67bc0f12e966 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:38.912375 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1294/2602] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:38.916733 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda282b-2583-4601-8927-c58ccf1692ad admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:38.932438 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda282b-2583-4601-8927-c58ccf1692ad admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:38.939183 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35fc28-37c2-4b0e-9d5e-a76de117bf68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:38.939886 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35fc28-37c2-4b0e-9d5e-a76de117bf68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:38.940194 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35fc28-37c2-4b0e-9d5e-a76de117bf68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:38.940474 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35fc28-37c2-4b0e-9d5e-a76de117bf68 admin admin] PATH_INFO: `/v3/projects/e0bf88d415944ee3a003f92e92682768/users/78b9010107e6492ba1e9ffaa1e200926/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:38.957504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35fc28-37c2-4b0e-9d5e-a76de117bf68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0bf88d415944ee3a003f92e92682768, user_id=78b9010107e6492ba1e9ffaa1e200926, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:38.960352 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35fc28-37c2-4b0e-9d5e-a76de117bf68 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:39.035488 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1309/2603] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:38 2022] PUT /identity/v3/projects/e0bf88d415944ee3a003f92e92682768/users/78b9010107e6492ba1e9ffaa1e200926/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:39.040154 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c69a06-85c9-476d-b638-67bc0f12e966 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:39.051389 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c69a06-85c9-476d-b638-67bc0f12e966 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:39.118292 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdf62d-9165-4a80-863e-1ffd39660101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:39.119409 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdf62d-9165-4a80-863e-1ffd39660101 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:39.119409 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdf62d-9165-4a80-863e-1ffd39660101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:39.119680 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdf62d-9165-4a80-863e-1ffd39660101 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:39.120124 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fdf62d-9165-4a80-863e-1ffd39660101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:39.121268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35fc28-37c2-4b0e-9d5e-a76de117bf68 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:39.121810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fdf62d-9165-4a80-863e-1ffd39660101 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:39.123584 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-0d35fc28-37c2-4b0e-9d5e-a76de117bf68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:39.136534 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1295/2604] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:39.146419 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdf62d-9165-4a80-863e-1ffd39660101 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:39.149124 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35fc28-37c2-4b0e-9d5e-a76de117bf68 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:39.157021 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8944bc-e1f3-43bd-9dfa-dadbdd398d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:39.157974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8944bc-e1f3-43bd-9dfa-dadbdd398d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:39.158268 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8944bc-e1f3-43bd-9dfa-dadbdd398d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:39.158659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8944bc-e1f3-43bd-9dfa-dadbdd398d16 admin admin] PATH_INFO: `/v3/projects/ea76fc5a846b4a9d98a862861fdbba4b` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:39.160170 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8944bc-e1f3-43bd-9dfa-dadbdd398d16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea76fc5a846b4a9d98a862861fdbba4b)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:39.161276 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdf62d-9165-4a80-863e-1ffd39660101 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:39.162241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8944bc-e1f3-43bd-9dfa-dadbdd398d16 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:39.167283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b445255-e1aa-4813-bbde-c037c1d68a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:39.167976 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b445255-e1aa-4813-bbde-c037c1d68a2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:39.170818 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b445255-e1aa-4813-bbde-c037c1d68a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:39.170818 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b445255-e1aa-4813-bbde-c037c1d68a2e admin admin] PATH_INFO: `/v3/projects/e0bf88d415944ee3a003f92e92682768/users/78b9010107e6492ba1e9ffaa1e200926/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:39.171723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b445255-e1aa-4813-bbde-c037c1d68a2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0bf88d415944ee3a003f92e92682768, user_id=78b9010107e6492ba1e9ffaa1e200926, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:39.176314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b445255-e1aa-4813-bbde-c037c1d68a2e admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:39.189860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1296/2605] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:39 2022] PUT /identity/v3/projects/e0bf88d415944ee3a003f92e92682768/users/78b9010107e6492ba1e9ffaa1e200926/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:39.259700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:39.357248 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5c8944bc-e1f3-43bd-9dfa-dadbdd398d16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea76fc5a846b4a9d98a862861fdbba4b', 'request_id': 'req-5c8944bc-e1f3-43bd-9dfa-dadbdd398d16'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:39.488109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5c8944bc-e1f3-43bd-9dfa-dadbdd398d16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea76fc5a846b4a9d98a862861fdbba4b', 'request_id': 'req-5c8944bc-e1f3-43bd-9dfa-dadbdd398d16'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:39.496979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5c8944bc-e1f3-43bd-9dfa-dadbdd398d16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea76fc5a846b4a9d98a862861fdbba4b', 'request_id': 'req-5c8944bc-e1f3-43bd-9dfa-dadbdd398d16'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:39.509273 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5c8944bc-e1f3-43bd-9dfa-dadbdd398d16 admin admin] The token cache is being invalidate because project ea76fc5a846b4a9d98a862861fdbba4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:39.509661 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5c8944bc-e1f3-43bd-9dfa-dadbdd398d16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:39.510250 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5c8944bc-e1f3-43bd-9dfa-dadbdd398d16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:39.511940 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1310/2606] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:39 2022] DELETE /identity/v3/projects/ea76fc5a846b4a9d98a862861fdbba4b => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:39.520448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b445255-e1aa-4813-bbde-c037c1d68a2e admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:39.524500 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-6b445255-e1aa-4813-bbde-c037c1d68a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:39.596514 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b445255-e1aa-4813-bbde-c037c1d68a2e admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:39.680682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043c73a-7678-4859-8b9b-2623d2e33068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:39.680682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043c73a-7678-4859-8b9b-2623d2e33068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:39.680682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043c73a-7678-4859-8b9b-2623d2e33068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:39.680682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043c73a-7678-4859-8b9b-2623d2e33068 admin admin] PATH_INFO: `/v3/users/154afa43db3349749366ebb7394c0e6c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:39.681561 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043c73a-7678-4859-8b9b-2623d2e33068 admin admin] RBAC: Authorizing `identity:delete_user(user_id=154afa43db3349749366ebb7394c0e6c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:39.682561 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043c73a-7678-4859-8b9b-2623d2e33068 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:40.029589 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-8043c73a-7678-4859-8b9b-2623d2e33068 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '154afa43db3349749366ebb7394c0e6c'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:40.158741 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-8043c73a-7678-4859-8b9b-2623d2e33068 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '154afa43db3349749366ebb7394c0e6c'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:40.159408 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-8043c73a-7678-4859-8b9b-2623d2e33068 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '154afa43db3349749366ebb7394c0e6c'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:40.167079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-8043c73a-7678-4859-8b9b-2623d2e33068 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '154afa43db3349749366ebb7394c0e6c'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:40.167814 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-8043c73a-7678-4859-8b9b-2623d2e33068 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '154afa43db3349749366ebb7394c0e6c'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:40.189185 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1297/2607] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:39 2022] DELETE /identity/v3/users/154afa43db3349749366ebb7394c0e6c => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:40.657570 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8944bc-e1f3-43bd-9dfa-dadbdd398d16 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:40.670258 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8944bc-e1f3-43bd-9dfa-dadbdd398d16 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:40.716600 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:40.736974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d478b8-a639-45f2-a19d-0ac29200dd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:40.737605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d478b8-a639-45f2-a19d-0ac29200dd8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:40.737877 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d478b8-a639-45f2-a19d-0ac29200dd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:40.738165 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d478b8-a639-45f2-a19d-0ac29200dd8c admin admin] PATH_INFO: `/v3/projects/f06a92e9e9da47dea03379108050202c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:40.739280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d478b8-a639-45f2-a19d-0ac29200dd8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f06a92e9e9da47dea03379108050202c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:40.741151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d478b8-a639-45f2-a19d-0ac29200dd8c admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:40.941872 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-e2d478b8-a639-45f2-a19d-0ac29200dd8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f06a92e9e9da47dea03379108050202c', 'request_id': 'req-e2d478b8-a639-45f2-a19d-0ac29200dd8c'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:41.018147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-e2d478b8-a639-45f2-a19d-0ac29200dd8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f06a92e9e9da47dea03379108050202c', 'request_id': 'req-e2d478b8-a639-45f2-a19d-0ac29200dd8c'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:41.027568 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-e2d478b8-a639-45f2-a19d-0ac29200dd8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f06a92e9e9da47dea03379108050202c', 'request_id': 'req-e2d478b8-a639-45f2-a19d-0ac29200dd8c'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:41.027962 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f225d35-fd6c-496b-85c5-72b0a2658741 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:41.028248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f225d35-fd6c-496b-85c5-72b0a2658741 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:41.028530 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f225d35-fd6c-496b-85c5-72b0a2658741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:41.047529 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-e2d478b8-a639-45f2-a19d-0ac29200dd8c admin admin] The token cache is being invalidate because project f06a92e9e9da47dea03379108050202c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:41.047529 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-e2d478b8-a639-45f2-a19d-0ac29200dd8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:41.047529 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-e2d478b8-a639-45f2-a19d-0ac29200dd8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:41.048329 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1311/2608] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:40 2022] DELETE /identity/v3/projects/f06a92e9e9da47dea03379108050202c => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:41.053159 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4a703-a5dc-45ff-8316-f20720269d1e None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:41.053439 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4a703-a5dc-45ff-8316-f20720269d1e None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:41.053716 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4a703-a5dc-45ff-8316-f20720269d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:41.061526 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-0f225d35-fd6c-496b-85c5-72b0a2658741 None None] MFA Rules not processed for user `78b9010107e6492ba1e9ffaa1e200926`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:41.063504 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-0f225d35-fd6c-496b-85c5-72b0a2658741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:41.073598 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:41.074340 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-71a4a703-a5dc-45ff-8316-f20720269d1e None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:41.075711 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-71a4a703-a5dc-45ff-8316-f20720269d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:41.170367 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1298/2609] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:41 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:41.193371 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1312/2610] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:41.206755 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf00cdb-dd1a-4e24-ab5c-c29bd6390e65 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:41.207216 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf00cdb-dd1a-4e24-ab5c-c29bd6390e65 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:41.209142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf00cdb-dd1a-4e24-ab5c-c29bd6390e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:41.233073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-5cf00cdb-dd1a-4e24-ab5c-c29bd6390e65 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:41.235448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-5cf00cdb-dd1a-4e24-ab5c-c29bd6390e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:41.244048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1299/2611] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:41.248562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a4a703-a5dc-45ff-8316-f20720269d1e None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:41.258901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a4a703-a5dc-45ff-8316-f20720269d1e None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:41.265930 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3e7f1-aadc-4458-8785-b914e3fa4c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:41.266489 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3e7f1-aadc-4458-8785-b914e3fa4c63 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:41.266802 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3e7f1-aadc-4458-8785-b914e3fa4c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:41.267077 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3e7f1-aadc-4458-8785-b914e3fa4c63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:41.267742 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa3e7f1-aadc-4458-8785-b914e3fa4c63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:41.268041 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa3e7f1-aadc-4458-8785-b914e3fa4c63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:41.269726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa3e7f1-aadc-4458-8785-b914e3fa4c63 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:41.288955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1313/2612] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:41.291029 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf00cdb-dd1a-4e24-ab5c-c29bd6390e65 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:41.313717 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf00cdb-dd1a-4e24-ab5c-c29bd6390e65 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:41.313717 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0f2cd-f7a2-41d7-9804-2da0c0a12e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:41.313717 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0f2cd-f7a2-41d7-9804-2da0c0a12e51 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:41.314304 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0f2cd-f7a2-41d7-9804-2da0c0a12e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:41.314491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0f2cd-f7a2-41d7-9804-2da0c0a12e51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:41.315271 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0f2cd-f7a2-41d7-9804-2da0c0a12e51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:41.320792 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0f2cd-f7a2-41d7-9804-2da0c0a12e51 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:41.404780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1300/2613] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:41 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:41.410261 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3e7f1-aadc-4458-8785-b914e3fa4c63 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:41.420932 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3e7f1-aadc-4458-8785-b914e3fa4c63 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:41.483784 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724c7bb-c61a-4aca-b494-e82c05bd5872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:41.484474 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724c7bb-c61a-4aca-b494-e82c05bd5872 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:41.484753 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724c7bb-c61a-4aca-b494-e82c05bd5872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:41.485143 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724c7bb-c61a-4aca-b494-e82c05bd5872 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:41.485774 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4724c7bb-c61a-4aca-b494-e82c05bd5872 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:41.487753 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4724c7bb-c61a-4aca-b494-e82c05bd5872 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:41.603969 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1314/2614] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:41 2022] POST /identity/v3/users => generated 292 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:41.608183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0f2cd-f7a2-41d7-9804-2da0c0a12e51 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:41.620290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0f2cd-f7a2-41d7-9804-2da0c0a12e51 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:41.625508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4798447f-42e4-47d5-8286-39f7c8c43090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:41.644666 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4798447f-42e4-47d5-8286-39f7c8c43090 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:41.644666 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4798447f-42e4-47d5-8286-39f7c8c43090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:41.644666 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4798447f-42e4-47d5-8286-39f7c8c43090 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:41.644666 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4798447f-42e4-47d5-8286-39f7c8c43090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:41.644666 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4798447f-42e4-47d5-8286-39f7c8c43090 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:41.644666 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1301/2615] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:41.645813 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724c7bb-c61a-4aca-b494-e82c05bd5872 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:41.655902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724c7bb-c61a-4aca-b494-e82c05bd5872 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:41.661149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b29b21-c053-434f-9981-4287491b0969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:41.661759 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b29b21-c053-434f-9981-4287491b0969 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:41.662023 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b29b21-c053-434f-9981-4287491b0969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:41.662289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b29b21-c053-434f-9981-4287491b0969 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:41.662982 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b29b21-c053-434f-9981-4287491b0969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:41.664572 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b29b21-c053-434f-9981-4287491b0969 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:41.678811 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1315/2616] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:41.681895 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4798447f-42e4-47d5-8286-39f7c8c43090 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:41.693205 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4798447f-42e4-47d5-8286-39f7c8c43090 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:41.697835 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96802d25-c9fb-464d-94bf-caa89f698385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:41.698535 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96802d25-c9fb-464d-94bf-caa89f698385 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:41.700384 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96802d25-c9fb-464d-94bf-caa89f698385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:41.700666 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96802d25-c9fb-464d-94bf-caa89f698385 admin admin] PATH_INFO: `/v3/projects/88e10af8c36f4dcc8408411b2ccdd52f/users/4348b5bb125a47c6b7a763b23f901689/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:41.723422 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96802d25-c9fb-464d-94bf-caa89f698385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88e10af8c36f4dcc8408411b2ccdd52f, user_id=4348b5bb125a47c6b7a763b23f901689, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:41.726783 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96802d25-c9fb-464d-94bf-caa89f698385 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:41.893682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1302/2617] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:41 2022] PUT /identity/v3/projects/88e10af8c36f4dcc8408411b2ccdd52f/users/4348b5bb125a47c6b7a763b23f901689/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:41.905150 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26243f-aa20-4a17-8386-00fed3821810 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:41.905549 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26243f-aa20-4a17-8386-00fed3821810 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:41.905833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26243f-aa20-4a17-8386-00fed3821810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:41.937860 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-0d26243f-aa20-4a17-8386-00fed3821810 None None] MFA Rules not processed for user `4348b5bb125a47c6b7a763b23f901689`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:41.939120 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-0d26243f-aa20-4a17-8386-00fed3821810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:42.009916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:42.020746 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1316/2618] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:42.025478 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96802d25-c9fb-464d-94bf-caa89f698385 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:42.094641 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96802d25-c9fb-464d-94bf-caa89f698385 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:42.100879 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3850cb-a220-4938-b636-22224085f592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:42.102048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3850cb-a220-4938-b636-22224085f592 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:42.102048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3850cb-a220-4938-b636-22224085f592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:42.102277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3850cb-a220-4938-b636-22224085f592 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:42.102874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3850cb-a220-4938-b636-22224085f592 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:42.104804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3850cb-a220-4938-b636-22224085f592 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:42.221039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1303/2619] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:42 2022] POST /identity/v3/projects => generated 381 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:42.225841 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26243f-aa20-4a17-8386-00fed3821810 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:42.242452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26243f-aa20-4a17-8386-00fed3821810 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:42.307361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9688864-daef-4c21-b6a1-7f72532e6fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:42.308084 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9688864-daef-4c21-b6a1-7f72532e6fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:42.308495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9688864-daef-4c21-b6a1-7f72532e6fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:42.308833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9688864-daef-4c21-b6a1-7f72532e6fc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:42.309519 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9688864-daef-4c21-b6a1-7f72532e6fc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:42.311857 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9688864-daef-4c21-b6a1-7f72532e6fc1 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:42.433813 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1317/2620] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:42 2022] POST /identity/v3/users => generated 297 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:42.439736 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3850cb-a220-4938-b636-22224085f592 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:42.454435 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3850cb-a220-4938-b636-22224085f592 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:42.460361 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d026b28-4ccd-4017-b97a-f9abe463da9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:42.461079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d026b28-4ccd-4017-b97a-f9abe463da9c admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:42.461422 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d026b28-4ccd-4017-b97a-f9abe463da9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:42.461761 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d026b28-4ccd-4017-b97a-f9abe463da9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:42.462464 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d026b28-4ccd-4017-b97a-f9abe463da9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:42.464170 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d026b28-4ccd-4017-b97a-f9abe463da9c admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:42.478229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1304/2621] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:42.482397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9688864-daef-4c21-b6a1-7f72532e6fc1 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:42.493553 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9688864-daef-4c21-b6a1-7f72532e6fc1 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:42.499231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307a4ee-7a2b-419e-81b5-552c407d5d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:42.500701 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307a4ee-7a2b-419e-81b5-552c407d5d86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:42.501259 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307a4ee-7a2b-419e-81b5-552c407d5d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:42.501651 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307a4ee-7a2b-419e-81b5-552c407d5d86 admin admin] PATH_INFO: `/v3/projects/ace50ce17dd54ba3868fcf401474d749/users/64d4f8e9c0b24553abeaf9d0691efa33/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:42.521648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307a4ee-7a2b-419e-81b5-552c407d5d86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ace50ce17dd54ba3868fcf401474d749, user_id=64d4f8e9c0b24553abeaf9d0691efa33, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:42.525674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307a4ee-7a2b-419e-81b5-552c407d5d86 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:42.712358 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1318/2622] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:42 2022] PUT /identity/v3/projects/ace50ce17dd54ba3868fcf401474d749/users/64d4f8e9c0b24553abeaf9d0691efa33/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:42.716515 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d026b28-4ccd-4017-b97a-f9abe463da9c admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:42.727703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d026b28-4ccd-4017-b97a-f9abe463da9c admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:42.740710 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:42.790294 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208f772-78f0-44f2-9907-061c5bc37500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:42.791300 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208f772-78f0-44f2-9907-061c5bc37500 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:42.791564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208f772-78f0-44f2-9907-061c5bc37500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:42.791843 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208f772-78f0-44f2-9907-061c5bc37500 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:42.792473 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208f772-78f0-44f2-9907-061c5bc37500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:42.794102 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208f772-78f0-44f2-9907-061c5bc37500 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:42.808935 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1305/2623] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:42.814645 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208f772-78f0-44f2-9907-061c5bc37500 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:42.814911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307a4ee-7a2b-419e-81b5-552c407d5d86 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:42.817142 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-1307a4ee-7a2b-419e-81b5-552c407d5d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:42.828546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208f772-78f0-44f2-9907-061c5bc37500 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:42.835365 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa68b1d-26ec-4b51-bed3-cde7a946f39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:42.836396 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa68b1d-26ec-4b51-bed3-cde7a946f39f admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:42.836875 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa68b1d-26ec-4b51-bed3-cde7a946f39f admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:42.837217 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa68b1d-26ec-4b51-bed3-cde7a946f39f admin admin] PATH_INFO: `/v3/projects/ace50ce17dd54ba3868fcf401474d749/users/64d4f8e9c0b24553abeaf9d0691efa33/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:42.838435 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307a4ee-7a2b-419e-81b5-552c407d5d86 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:42.839484 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa68b1d-26ec-4b51-bed3-cde7a946f39f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ace50ce17dd54ba3868fcf401474d749, user_id=64d4f8e9c0b24553abeaf9d0691efa33, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:42.842387 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa68b1d-26ec-4b51-bed3-cde7a946f39f admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:42.846176 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3f94e-2b78-426a-b04e-3b4b84049f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:42.848292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3f94e-2b78-426a-b04e-3b4b84049f34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:42.848292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3f94e-2b78-426a-b04e-3b4b84049f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:42.848292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3f94e-2b78-426a-b04e-3b4b84049f34 admin admin] PATH_INFO: `/v3/users/9963f7d305294acc841ad567899383cf` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:42.849412 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3f94e-2b78-426a-b04e-3b4b84049f34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9963f7d305294acc841ad567899383cf)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:42.851618 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3f94e-2b78-426a-b04e-3b4b84049f34 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:42.855893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1306/2624] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:42 2022] PUT /identity/v3/projects/ace50ce17dd54ba3868fcf401474d749/users/64d4f8e9c0b24553abeaf9d0691efa33/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:42.867135 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf5a61-8851-40bb-932e-6f05b7b0c95e None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:42.867754 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf5a61-8851-40bb-932e-6f05b7b0c95e None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:42.868033 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf5a61-8851-40bb-932e-6f05b7b0c95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:42.901389 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-5edf5a61-8851-40bb-932e-6f05b7b0c95e None None] MFA Rules not processed for user `64d4f8e9c0b24553abeaf9d0691efa33`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:42.902662 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-5edf5a61-8851-40bb-932e-6f05b7b0c95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:42.974586 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=89,keystone:INSERT=7,keystone:DELETE=4 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:42.986764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1307/2625] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:42 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:42.996105 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf5a61-8851-40bb-932e-6f05b7b0c95e None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:42.998102 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-5edf5a61-8851-40bb-932e-6f05b7b0c95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:43.042078 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b0c3f94e-2b78-426a-b04e-3b4b84049f34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9963f7d305294acc841ad567899383cf'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:43.067816 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf5a61-8851-40bb-932e-6f05b7b0c95e None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:43.071603 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2c61f-f65d-4e0e-8734-8e6842a60e1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:43.072215 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2c61f-f65d-4e0e-8734-8e6842a60e1d service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:43.072495 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2c61f-f65d-4e0e-8734-8e6842a60e1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:43.072765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2c61f-f65d-4e0e-8734-8e6842a60e1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:43.083220 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2c61f-f65d-4e0e-8734-8e6842a60e1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:43.084659 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2c61f-f65d-4e0e-8734-8e6842a60e1d service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:43.095242 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1308/2626] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:42 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:43.111846 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b0c3f94e-2b78-426a-b04e-3b4b84049f34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9963f7d305294acc841ad567899383cf'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:43.112604 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b0c3f94e-2b78-426a-b04e-3b4b84049f34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9963f7d305294acc841ad567899383cf'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:43.120327 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b0c3f94e-2b78-426a-b04e-3b4b84049f34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9963f7d305294acc841ad567899383cf'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:43.133993 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b0c3f94e-2b78-426a-b04e-3b4b84049f34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9963f7d305294acc841ad567899383cf'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:43.141588 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1319/2627] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:42 2022] DELETE /identity/v3/users/9963f7d305294acc841ad567899383cf => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:43.271772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2c61f-f65d-4e0e-8734-8e6842a60e1d service cinder] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:43.285036 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2c61f-f65d-4e0e-8734-8e6842a60e1d service cinder] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:43.319348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:43.343063 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d19a4c-2001-47b5-a484-8299b6cc3667 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:43.347621 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d19a4c-2001-47b5-a484-8299b6cc3667 service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:43.347621 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d19a4c-2001-47b5-a484-8299b6cc3667 service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:43.347621 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d19a4c-2001-47b5-a484-8299b6cc3667 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:43.349917 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-04d19a4c-2001-47b5-a484-8299b6cc3667 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:43.403612 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3f94e-2b78-426a-b04e-3b4b84049f34 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:43.405896 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b0c3f94e-2b78-426a-b04e-3b4b84049f34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:43.431313 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d19a4c-2001-47b5-a484-8299b6cc3667 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:43.432947 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d19a4c-2001-47b5-a484-8299b6cc3667 service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:43.454705 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1309/2628] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:43.526232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3f94e-2b78-426a-b04e-3b4b84049f34 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:43.533415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acbecf-d93a-4d1a-a617-26cf5aa3c151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:43.534091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acbecf-d93a-4d1a-a617-26cf5aa3c151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:43.534367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acbecf-d93a-4d1a-a617-26cf5aa3c151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:43.539132 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acbecf-d93a-4d1a-a617-26cf5aa3c151 admin admin] PATH_INFO: `/v3/projects/3ac2d654c4ba4073861b517518a3b5d4` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:43.541549 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43acbecf-d93a-4d1a-a617-26cf5aa3c151 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ac2d654c4ba4073861b517518a3b5d4)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:43.542632 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43acbecf-d93a-4d1a-a617-26cf5aa3c151 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:43.720152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-43acbecf-d93a-4d1a-a617-26cf5aa3c151 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ac2d654c4ba4073861b517518a3b5d4', 'request_id': 'req-43acbecf-d93a-4d1a-a617-26cf5aa3c151'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:43.803625 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-43acbecf-d93a-4d1a-a617-26cf5aa3c151 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ac2d654c4ba4073861b517518a3b5d4', 'request_id': 'req-43acbecf-d93a-4d1a-a617-26cf5aa3c151'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:43.811939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-43acbecf-d93a-4d1a-a617-26cf5aa3c151 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ac2d654c4ba4073861b517518a3b5d4', 'request_id': 'req-43acbecf-d93a-4d1a-a617-26cf5aa3c151'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:43.826260 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-43acbecf-d93a-4d1a-a617-26cf5aa3c151 admin admin] The token cache is being invalidate because project 3ac2d654c4ba4073861b517518a3b5d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:43.826666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-43acbecf-d93a-4d1a-a617-26cf5aa3c151 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:43.827425 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-43acbecf-d93a-4d1a-a617-26cf5aa3c151 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:43.830112 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1320/2629] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:43 2022] DELETE /identity/v3/projects/3ac2d654c4ba4073861b517518a3b5d4 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:43.834384 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d19a4c-2001-47b5-a484-8299b6cc3667 service cinder] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:43.846273 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d19a4c-2001-47b5-a484-8299b6cc3667 service cinder] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:43.936259 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554ed77-b205-42c8-b315-0db235873f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:43.937561 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554ed77-b205-42c8-b315-0db235873f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:43.937561 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554ed77-b205-42c8-b315-0db235873f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:43.937809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554ed77-b205-42c8-b315-0db235873f74 admin admin] PATH_INFO: `/v3/users/78b9010107e6492ba1e9ffaa1e200926` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:43.939121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554ed77-b205-42c8-b315-0db235873f74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b9010107e6492ba1e9ffaa1e200926)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:43.941250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554ed77-b205-42c8-b315-0db235873f74 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:44.097132 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acbecf-d93a-4d1a-a617-26cf5aa3c151 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:44.105582 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-43acbecf-d93a-4d1a-a617-26cf5aa3c151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:44.124362 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acbecf-d93a-4d1a-a617-26cf5aa3c151 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:44.131346 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffb39e-793a-4ae4-8f7b-8b9b0095bde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:44.132170 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffb39e-793a-4ae4-8f7b-8b9b0095bde4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:44.132469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffb39e-793a-4ae4-8f7b-8b9b0095bde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:44.132748 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffb39e-793a-4ae4-8f7b-8b9b0095bde4 admin admin] PATH_INFO: `/v3/users/4348b5bb125a47c6b7a763b23f901689` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:44.134956 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cffb39e-793a-4ae4-8f7b-8b9b0095bde4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4348b5bb125a47c6b7a763b23f901689)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:44.137152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cffb39e-793a-4ae4-8f7b-8b9b0095bde4 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:44.164327 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f554ed77-b205-42c8-b315-0db235873f74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b9010107e6492ba1e9ffaa1e200926'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:44.270683 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f554ed77-b205-42c8-b315-0db235873f74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b9010107e6492ba1e9ffaa1e200926'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:44.273995 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f554ed77-b205-42c8-b315-0db235873f74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b9010107e6492ba1e9ffaa1e200926'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:44.291183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f554ed77-b205-42c8-b315-0db235873f74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b9010107e6492ba1e9ffaa1e200926'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:44.298693 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f554ed77-b205-42c8-b315-0db235873f74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b9010107e6492ba1e9ffaa1e200926'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:44.308240 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:44.313356 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1310/2630] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:43 2022] DELETE /identity/v3/users/78b9010107e6492ba1e9ffaa1e200926 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:44.319516 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:44.430251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5cffb39e-793a-4ae4-8f7b-8b9b0095bde4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4348b5bb125a47c6b7a763b23f901689'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:44.523137 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5cffb39e-793a-4ae4-8f7b-8b9b0095bde4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4348b5bb125a47c6b7a763b23f901689'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:44.523853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5cffb39e-793a-4ae4-8f7b-8b9b0095bde4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4348b5bb125a47c6b7a763b23f901689'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:44.532606 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5cffb39e-793a-4ae4-8f7b-8b9b0095bde4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4348b5bb125a47c6b7a763b23f901689'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:44.545899 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5cffb39e-793a-4ae4-8f7b-8b9b0095bde4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4348b5bb125a47c6b7a763b23f901689'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:44.552750 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1321/2631] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:44 2022] DELETE /identity/v3/users/4348b5bb125a47c6b7a763b23f901689 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:44.557609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6a411-05fe-4f11-b9e0-88d9c7717201 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:44.557896 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6a411-05fe-4f11-b9e0-88d9c7717201 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:44.558222 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6a411-05fe-4f11-b9e0-88d9c7717201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:44.580099 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-ddb6a411-05fe-4f11-b9e0-88d9c7717201 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:44.581248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ddb6a411-05fe-4f11-b9e0-88d9c7717201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:44.659166 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1311/2632] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:44.665955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffb39e-793a-4ae4-8f7b-8b9b0095bde4 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:44.667471 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-5cffb39e-793a-4ae4-8f7b-8b9b0095bde4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:44.720128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6a411-05fe-4f11-b9e0-88d9c7717201 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:44.722062 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ddb6a411-05fe-4f11-b9e0-88d9c7717201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:44.737818 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffb39e-793a-4ae4-8f7b-8b9b0095bde4 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:44.742014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267e4e1-ab9f-4ec8-8ea0-7b877163de58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:44.743049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267e4e1-ab9f-4ec8-8ea0-7b877163de58 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:44.743426 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267e4e1-ab9f-4ec8-8ea0-7b877163de58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:44.743836 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267e4e1-ab9f-4ec8-8ea0-7b877163de58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:44.744842 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6a411-05fe-4f11-b9e0-88d9c7717201 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:44.751193 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34323764-aff9-4d68-b9ab-8c25ba37e0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:44.751939 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34323764-aff9-4d68-b9ab-8c25ba37e0ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:44.752380 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34323764-aff9-4d68-b9ab-8c25ba37e0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:44.752708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34323764-aff9-4d68-b9ab-8c25ba37e0ce admin admin] PATH_INFO: `/v3/projects/e0bf88d415944ee3a003f92e92682768` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:44.753897 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34323764-aff9-4d68-b9ab-8c25ba37e0ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0bf88d415944ee3a003f92e92682768)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:44.755932 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267e4e1-ab9f-4ec8-8ea0-7b877163de58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:44.756085 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34323764-aff9-4d68-b9ab-8c25ba37e0ce admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:44.757473 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267e4e1-ab9f-4ec8-8ea0-7b877163de58 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:44.768324 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1322/2633] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:44.907525 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-34323764-aff9-4d68-b9ab-8c25ba37e0ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0bf88d415944ee3a003f92e92682768', 'request_id': 'req-34323764-aff9-4d68-b9ab-8c25ba37e0ce'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:44.912825 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-34323764-aff9-4d68-b9ab-8c25ba37e0ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0bf88d415944ee3a003f92e92682768', 'request_id': 'req-34323764-aff9-4d68-b9ab-8c25ba37e0ce'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:44.919570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-34323764-aff9-4d68-b9ab-8c25ba37e0ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0bf88d415944ee3a003f92e92682768', 'request_id': 'req-34323764-aff9-4d68-b9ab-8c25ba37e0ce'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:45.052390 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-34323764-aff9-4d68-b9ab-8c25ba37e0ce admin admin] The token cache is being invalidate because project e0bf88d415944ee3a003f92e92682768 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:45.052390 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-34323764-aff9-4d68-b9ab-8c25ba37e0ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:45.052390 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-34323764-aff9-4d68-b9ab-8c25ba37e0ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:45.052390 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1312/2634] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:44 2022] DELETE /identity/v3/projects/e0bf88d415944ee3a003f92e92682768 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:45.056486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67038451-2f44-48c7-9b76-68a0c5fe085a None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:45.056720 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67038451-2f44-48c7-9b76-68a0c5fe085a None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:45.057100 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67038451-2f44-48c7-9b76-68a0c5fe085a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:45.079174 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-67038451-2f44-48c7-9b76-68a0c5fe085a None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:45.080391 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-67038451-2f44-48c7-9b76-68a0c5fe085a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:45.141256 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1323/2635] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:45.149090 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f01739-9f9b-4f1c-a9eb-44323ee34e4a None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:45.149887 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f01739-9f9b-4f1c-a9eb-44323ee34e4a None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:45.150226 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f01739-9f9b-4f1c-a9eb-44323ee34e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:45.170528 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-28f01739-9f9b-4f1c-a9eb-44323ee34e4a None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:45.172380 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-28f01739-9f9b-4f1c-a9eb-44323ee34e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:45.180367 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1313/2636] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:45.185060 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67038451-2f44-48c7-9b76-68a0c5fe085a None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:45.196314 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67038451-2f44-48c7-9b76-68a0c5fe085a None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:45.201117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66259395-cbd9-4784-963a-b797eeaafce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:45.201735 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66259395-cbd9-4784-963a-b797eeaafce1 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:45.202012 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66259395-cbd9-4784-963a-b797eeaafce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:45.202278 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66259395-cbd9-4784-963a-b797eeaafce1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:45.203218 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66259395-cbd9-4784-963a-b797eeaafce1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:45.203525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66259395-cbd9-4784-963a-b797eeaafce1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:45.205111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66259395-cbd9-4784-963a-b797eeaafce1 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:45.216873 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1324/2637] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:45.222117 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f01739-9f9b-4f1c-a9eb-44323ee34e4a None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:45.231411 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f01739-9f9b-4f1c-a9eb-44323ee34e4a None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:45.236706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035fc3cc-9453-485b-8fbc-e68eaad37019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:45.237297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fc3cc-9453-485b-8fbc-e68eaad37019 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:45.238131 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fc3cc-9453-485b-8fbc-e68eaad37019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:45.238398 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fc3cc-9453-485b-8fbc-e68eaad37019 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:45.239070 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035fc3cc-9453-485b-8fbc-e68eaad37019 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:45.241074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035fc3cc-9453-485b-8fbc-e68eaad37019 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:45.357254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66259395-cbd9-4784-963a-b797eeaafce1 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:45.360519 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-66259395-cbd9-4784-963a-b797eeaafce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:45.387830 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1314/2638] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:45 2022] POST /identity/v3/projects => generated 369 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:45.392092 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66259395-cbd9-4784-963a-b797eeaafce1 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:45.397342 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035fc3cc-9453-485b-8fbc-e68eaad37019 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:45.436432 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:45.494291 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a2f17-765d-4738-b312-62f46b5a8d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:45.497819 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a2f17-765d-4738-b312-62f46b5a8d68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:45.500503 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a2f17-765d-4738-b312-62f46b5a8d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:45.500839 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a2f17-765d-4738-b312-62f46b5a8d68 admin admin] PATH_INFO: `/v3/projects/88e10af8c36f4dcc8408411b2ccdd52f` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:45.505929 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a2f17-765d-4738-b312-62f46b5a8d68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88e10af8c36f4dcc8408411b2ccdd52f)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:45.509410 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a2f17-765d-4738-b312-62f46b5a8d68 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:45.525461 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:45.548030 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035fc3cc-9453-485b-8fbc-e68eaad37019 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:45.553193 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b4458-188b-4f02-b373-b709e825688e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:45.553859 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b4458-188b-4f02-b373-b709e825688e admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:45.554049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b4458-188b-4f02-b373-b709e825688e admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:45.554049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b4458-188b-4f02-b373-b709e825688e admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:45.554962 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b4458-188b-4f02-b373-b709e825688e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:45.557162 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b4458-188b-4f02-b373-b709e825688e admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:45.682490 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1315/2639] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:45 2022] POST /identity/v3/users => generated 285 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:45.682913 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-df5a2f17-765d-4738-b312-62f46b5a8d68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88e10af8c36f4dcc8408411b2ccdd52f', 'request_id': 'req-df5a2f17-765d-4738-b312-62f46b5a8d68'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:45.687799 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b4458-188b-4f02-b373-b709e825688e admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:45.697624 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b4458-188b-4f02-b373-b709e825688e admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:45.754445 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afb0ef-9a6d-4c6a-91b2-bc361813ddfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:45.755094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afb0ef-9a6d-4c6a-91b2-bc361813ddfb admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:45.755369 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afb0ef-9a6d-4c6a-91b2-bc361813ddfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:45.755638 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afb0ef-9a6d-4c6a-91b2-bc361813ddfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:45.756282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0afb0ef-9a6d-4c6a-91b2-bc361813ddfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:45.758271 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0afb0ef-9a6d-4c6a-91b2-bc361813ddfb admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:45.769291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1316/2640] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:45.773378 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afb0ef-9a6d-4c6a-91b2-bc361813ddfb admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:45.783246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afb0ef-9a6d-4c6a-91b2-bc361813ddfb admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:45.787786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28609cd8-4e4e-4dc3-ae0a-88975d7a5c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:45.788446 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28609cd8-4e4e-4dc3-ae0a-88975d7a5c74 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:45.788989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28609cd8-4e4e-4dc3-ae0a-88975d7a5c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:45.789262 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28609cd8-4e4e-4dc3-ae0a-88975d7a5c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:45.790002 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28609cd8-4e4e-4dc3-ae0a-88975d7a5c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:45.790279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-df5a2f17-765d-4738-b312-62f46b5a8d68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88e10af8c36f4dcc8408411b2ccdd52f', 'request_id': 'req-df5a2f17-765d-4738-b312-62f46b5a8d68'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:45.791653 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28609cd8-4e4e-4dc3-ae0a-88975d7a5c74 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:45.799973 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-df5a2f17-765d-4738-b312-62f46b5a8d68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88e10af8c36f4dcc8408411b2ccdd52f', 'request_id': 'req-df5a2f17-765d-4738-b312-62f46b5a8d68'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:45.804798 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1317/2641] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:45.807948 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28609cd8-4e4e-4dc3-ae0a-88975d7a5c74 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:45.813675 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-df5a2f17-765d-4738-b312-62f46b5a8d68 admin admin] The token cache is being invalidate because project 88e10af8c36f4dcc8408411b2ccdd52f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:45.813967 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-df5a2f17-765d-4738-b312-62f46b5a8d68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:45.814727 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-df5a2f17-765d-4738-b312-62f46b5a8d68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:45.816363 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1325/2642] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:45 2022] DELETE /identity/v3/projects/88e10af8c36f4dcc8408411b2ccdd52f => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:45.819508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28609cd8-4e4e-4dc3-ae0a-88975d7a5c74 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:45.820208 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a2f17-765d-4738-b312-62f46b5a8d68 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:45.821525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-df5a2f17-765d-4738-b312-62f46b5a8d68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:45.824353 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9c0bc-7f13-4663-a9ed-c922672e18b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:45.825112 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9c0bc-7f13-4663-a9ed-c922672e18b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:45.825576 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9c0bc-7f13-4663-a9ed-c922672e18b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:45.825974 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9c0bc-7f13-4663-a9ed-c922672e18b7 admin admin] PATH_INFO: `/v3/projects/c8c176f660e8470eb128102c1f7aa8d0/users/a4e78c3d7a1c4ee49c3755298f132951/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:45.834444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a2f17-765d-4738-b312-62f46b5a8d68 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:45.839069 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383bcd19-0878-4553-8097-74323cdbcc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:45.839693 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bcd19-0878-4553-8097-74323cdbcc7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:45.839965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bcd19-0878-4553-8097-74323cdbcc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:45.840235 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bcd19-0878-4553-8097-74323cdbcc7f admin admin] PATH_INFO: `/v3/users/64d4f8e9c0b24553abeaf9d0691efa33` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:45.841260 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9c0bc-7f13-4663-a9ed-c922672e18b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8c176f660e8470eb128102c1f7aa8d0, user_id=a4e78c3d7a1c4ee49c3755298f132951, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:45.841389 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383bcd19-0878-4553-8097-74323cdbcc7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=64d4f8e9c0b24553abeaf9d0691efa33)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:45.843223 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383bcd19-0878-4553-8097-74323cdbcc7f admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:45.844031 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9c0bc-7f13-4663-a9ed-c922672e18b7 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:45.917431 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1318/2643] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:45 2022] PUT /identity/v3/projects/c8c176f660e8470eb128102c1f7aa8d0/users/a4e78c3d7a1c4ee49c3755298f132951/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:45.930004 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d9321-e3e4-4089-ae45-16cea78335a0 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:45.930336 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d9321-e3e4-4089-ae45-16cea78335a0 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:45.930727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d9321-e3e4-4089-ae45-16cea78335a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:45.961416 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-cf2d9321-e3e4-4089-ae45-16cea78335a0 None None] MFA Rules not processed for user `a4e78c3d7a1c4ee49c3755298f132951`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:45.962641 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-cf2d9321-e3e4-4089-ae45-16cea78335a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:46.044865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1319/2644] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:46.050522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d9321-e3e4-4089-ae45-16cea78335a0 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:46.052013 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-cf2d9321-e3e4-4089-ae45-16cea78335a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:46.076664 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-383bcd19-0878-4553-8097-74323cdbcc7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64d4f8e9c0b24553abeaf9d0691efa33'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:46.090816 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:46.126487 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d9321-e3e4-4089-ae45-16cea78335a0 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:46.131328 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a447029-f6fb-443e-badd-18ec4c68a4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:46.131931 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a447029-f6fb-443e-badd-18ec4c68a4af admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:46.132203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a447029-f6fb-443e-badd-18ec4c68a4af admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:46.132467 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a447029-f6fb-443e-badd-18ec4c68a4af admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:46.133084 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a447029-f6fb-443e-badd-18ec4c68a4af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:46.136053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a447029-f6fb-443e-badd-18ec4c68a4af admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:46.158095 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-383bcd19-0878-4553-8097-74323cdbcc7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64d4f8e9c0b24553abeaf9d0691efa33'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:46.158938 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-383bcd19-0878-4553-8097-74323cdbcc7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64d4f8e9c0b24553abeaf9d0691efa33'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:46.166802 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-383bcd19-0878-4553-8097-74323cdbcc7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64d4f8e9c0b24553abeaf9d0691efa33'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:46.180466 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-383bcd19-0878-4553-8097-74323cdbcc7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64d4f8e9c0b24553abeaf9d0691efa33'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:46.191768 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1326/2645] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:45 2022] DELETE /identity/v3/users/64d4f8e9c0b24553abeaf9d0691efa33 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:46.271542 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1320/2646] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:46 2022] POST /identity/v3/projects => generated 369 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:46.272817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383bcd19-0878-4553-8097-74323cdbcc7f admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:46.274887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-383bcd19-0878-4553-8097-74323cdbcc7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:46.313474 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40976e7-7e73-4484-b60d-fbe8da57ab16 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:46.314080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40976e7-7e73-4484-b60d-fbe8da57ab16 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:46.314997 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40976e7-7e73-4484-b60d-fbe8da57ab16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:46.334150 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:46.336956 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-f40976e7-7e73-4484-b60d-fbe8da57ab16 None None] MFA Rules not processed for user `9646662b0b3048f79ad5d2b9602f10a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:46.338210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-f40976e7-7e73-4484-b60d-fbe8da57ab16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:46.357916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383bcd19-0878-4553-8097-74323cdbcc7f admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:46.405681 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1321/2647] 10.222.0.13 () {58 vars in 967 bytes} [Mon Aug 8 17:01:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:46.419313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82d5e9-40c2-4c47-8143-d52e3c324d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:46.419640 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40976e7-7e73-4484-b60d-fbe8da57ab16 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:46.420089 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82d5e9-40c2-4c47-8143-d52e3c324d33 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:46.420435 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82d5e9-40c2-4c47-8143-d52e3c324d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:46.420777 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82d5e9-40c2-4c47-8143-d52e3c324d33 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:46.421053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-f40976e7-7e73-4484-b60d-fbe8da57ab16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:46.421631 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82d5e9-40c2-4c47-8143-d52e3c324d33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:46.423736 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82d5e9-40c2-4c47-8143-d52e3c324d33 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:46.505415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40976e7-7e73-4484-b60d-fbe8da57ab16 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:46.510860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12875554-4cbc-4a96-afea-432b56826f5a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c606decb832444e38a194b98f8d15f8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:46.511933 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12875554-4cbc-4a96-afea-432b56826f5a service swift] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:46.511933 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12875554-4cbc-4a96-afea-432b56826f5a service swift] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:46.512210 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12875554-4cbc-4a96-afea-432b56826f5a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:46.521976 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12875554-4cbc-4a96-afea-432b56826f5a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:46.524182 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12875554-4cbc-4a96-afea-432b56826f5a service swift] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:46.533564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1322/2648] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Aug 8 17:01:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 17:01:46.565760 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1327/2649] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:46 2022] POST /identity/v3/users => generated 291 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:46.568242 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12875554-4cbc-4a96-afea-432b56826f5a service swift] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:46.580826 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12875554-4cbc-4a96-afea-432b56826f5a service swift] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:46.591989 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949b039-3aa3-48ef-8d29-a8fb8d48fa45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:46.592618 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949b039-3aa3-48ef-8d29-a8fb8d48fa45 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:46.592902 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949b039-3aa3-48ef-8d29-a8fb8d48fa45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:46.593174 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949b039-3aa3-48ef-8d29-a8fb8d48fa45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:46.594501 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d949b039-3aa3-48ef-8d29-a8fb8d48fa45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:46.598277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d949b039-3aa3-48ef-8d29-a8fb8d48fa45 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:46.616860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1323/2650] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:46.621447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82d5e9-40c2-4c47-8143-d52e3c324d33 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:46.638041 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82d5e9-40c2-4c47-8143-d52e3c324d33 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:46.645393 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49aad46-d77e-46d1-96ff-a094d903819f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:46.647259 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49aad46-d77e-46d1-96ff-a094d903819f admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:46.647542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49aad46-d77e-46d1-96ff-a094d903819f admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:46.647817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49aad46-d77e-46d1-96ff-a094d903819f admin admin] PATH_INFO: `/v3/projects/948bf3d813dd493abda63120d53952d5/users/4eb7018c3c3142d784f0d49b3bfa151f/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:46.655290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949b039-3aa3-48ef-8d29-a8fb8d48fa45 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:46.657327 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d949b039-3aa3-48ef-8d29-a8fb8d48fa45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:46.666443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49aad46-d77e-46d1-96ff-a094d903819f admin admin] RBAC: Authorizing `identity:create_grant(project_id=948bf3d813dd493abda63120d53952d5, user_id=4eb7018c3c3142d784f0d49b3bfa151f, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:46.669270 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49aad46-d77e-46d1-96ff-a094d903819f admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:46.705177 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:46.723779 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949b039-3aa3-48ef-8d29-a8fb8d48fa45 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:46.727617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e793b666-f628-428d-b905-a845c139f303 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:46.728272 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793b666-f628-428d-b905-a845c139f303 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:46.728544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793b666-f628-428d-b905-a845c139f303 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:46.728809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793b666-f628-428d-b905-a845c139f303 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:46.730624 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-e793b666-f628-428d-b905-a845c139f303 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:46.758905 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1328/2651] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:46 2022] PUT /identity/v3/projects/948bf3d813dd493abda63120d53952d5/users/4eb7018c3c3142d784f0d49b3bfa151f/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:46.759889 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49aad46-d77e-46d1-96ff-a094d903819f admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:46.761901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-a49aad46-d77e-46d1-96ff-a094d903819f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:46.840090 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e793b666-f628-428d-b905-a845c139f303 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:46.841427 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49aad46-d77e-46d1-96ff-a094d903819f admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:46.842719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e793b666-f628-428d-b905-a845c139f303 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:46.845744 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e8bb0-eece-471f-b52d-01ec7bdfebad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:46.847029 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e8bb0-eece-471f-b52d-01ec7bdfebad service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:46.847367 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e8bb0-eece-471f-b52d-01ec7bdfebad service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:46.847848 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e8bb0-eece-471f-b52d-01ec7bdfebad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:46.849690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-416e8bb0-eece-471f-b52d-01ec7bdfebad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:46.864942 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1324/2652] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:46.867512 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e793b666-f628-428d-b905-a845c139f303 service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:46.879071 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e793b666-f628-428d-b905-a845c139f303 service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:46.938690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:46.960967 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e8bb0-eece-471f-b52d-01ec7bdfebad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:46.962794 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e8bb0-eece-471f-b52d-01ec7bdfebad service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:46.973504 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5b6b1-ad78-4693-a2b1-411aaa73e8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:46.975139 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5b6b1-ad78-4693-a2b1-411aaa73e8b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:46.975420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5b6b1-ad78-4693-a2b1-411aaa73e8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:46.975953 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5b6b1-ad78-4693-a2b1-411aaa73e8b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:46.976607 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5b6b1-ad78-4693-a2b1-411aaa73e8b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:46.977982 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1329/2653] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:46.979732 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5b6b1-ad78-4693-a2b1-411aaa73e8b7 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:46.990140 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e8bb0-eece-471f-b52d-01ec7bdfebad service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:46.993676 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-416e8bb0-eece-471f-b52d-01ec7bdfebad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:46.996541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1325/2654] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:47.009695 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e8bb0-eece-471f-b52d-01ec7bdfebad service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:47.012030 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5b6b1-ad78-4693-a2b1-411aaa73e8b7 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:47.020409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4368c64-cf8e-435c-9644-a36bde4ac028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:47.021272 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4368c64-cf8e-435c-9644-a36bde4ac028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:47.021660 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4368c64-cf8e-435c-9644-a36bde4ac028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:47.022006 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4368c64-cf8e-435c-9644-a36bde4ac028 admin admin] PATH_INFO: `/v3/projects/ace50ce17dd54ba3868fcf401474d749` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:47.023738 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4368c64-cf8e-435c-9644-a36bde4ac028 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ace50ce17dd54ba3868fcf401474d749)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:47.026285 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4368c64-cf8e-435c-9644-a36bde4ac028 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:47.039999 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5b6b1-ad78-4693-a2b1-411aaa73e8b7 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:47.045601 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3bf86-763c-4f0b-a6b5-f6ac805c5efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:47.046327 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3bf86-763c-4f0b-a6b5-f6ac805c5efd admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:47.046723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3bf86-763c-4f0b-a6b5-f6ac805c5efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:47.046996 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3bf86-763c-4f0b-a6b5-f6ac805c5efd admin admin] PATH_INFO: `/v3/projects/948bf3d813dd493abda63120d53952d5/users/4eb7018c3c3142d784f0d49b3bfa151f/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:47.049051 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3bf86-763c-4f0b-a6b5-f6ac805c5efd admin admin] RBAC: Authorizing `identity:create_grant(project_id=948bf3d813dd493abda63120d53952d5, user_id=4eb7018c3c3142d784f0d49b3bfa151f, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:47.058871 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3bf86-763c-4f0b-a6b5-f6ac805c5efd admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:47.073503 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1326/2655] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:47 2022] PUT /identity/v3/projects/948bf3d813dd493abda63120d53952d5/users/4eb7018c3c3142d784f0d49b3bfa151f/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:47.094231 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5d31f-397d-4b70-b192-e5ddc348667e None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:47.094715 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5d31f-397d-4b70-b192-e5ddc348667e None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:47.095058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5d31f-397d-4b70-b192-e5ddc348667e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:47.123792 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-92d5d31f-397d-4b70-b192-e5ddc348667e None None] MFA Rules not processed for user `4eb7018c3c3142d784f0d49b3bfa151f`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:47.125097 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-92d5d31f-397d-4b70-b192-e5ddc348667e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:47.153889 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:47.201043 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1327/2656] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:47 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:47.208395 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5d31f-397d-4b70-b192-e5ddc348667e None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:47.209809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-92d5d31f-397d-4b70-b192-e5ddc348667e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:47.241118 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-a4368c64-cf8e-435c-9644-a36bde4ac028 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ace50ce17dd54ba3868fcf401474d749', 'request_id': 'req-a4368c64-cf8e-435c-9644-a36bde4ac028'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:47.273048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5d31f-397d-4b70-b192-e5ddc348667e None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:47.324817 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c0552-1f47-40c3-9fc1-7e86b1c2f04e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:47.325416 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c0552-1f47-40c3-9fc1-7e86b1c2f04e service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:47.325695 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c0552-1f47-40c3-9fc1-7e86b1c2f04e service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:47.326075 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c0552-1f47-40c3-9fc1-7e86b1c2f04e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:47.363633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-a4368c64-cf8e-435c-9644-a36bde4ac028 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ace50ce17dd54ba3868fcf401474d749', 'request_id': 'req-a4368c64-cf8e-435c-9644-a36bde4ac028'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:47.370969 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-a4368c64-cf8e-435c-9644-a36bde4ac028 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ace50ce17dd54ba3868fcf401474d749', 'request_id': 'req-a4368c64-cf8e-435c-9644-a36bde4ac028'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:47.382926 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-a4368c64-cf8e-435c-9644-a36bde4ac028 admin admin] The token cache is being invalidate because project ace50ce17dd54ba3868fcf401474d749 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:47.383210 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-a4368c64-cf8e-435c-9644-a36bde4ac028 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:47.384035 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-a4368c64-cf8e-435c-9644-a36bde4ac028 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:47.385534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1330/2657] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:46 2022] DELETE /identity/v3/projects/ace50ce17dd54ba3868fcf401474d749 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:47.390846 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edff42-b71b-411b-9129-5b6431e4bbc9 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:47.391196 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edff42-b71b-411b-9129-5b6431e4bbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:47.391483 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edff42-b71b-411b-9129-5b6431e4bbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:47.399318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c0552-1f47-40c3-9fc1-7e86b1c2f04e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:47.400914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c0552-1f47-40c3-9fc1-7e86b1c2f04e service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:47.411046 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1328/2658] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:47.415384 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-a3edff42-b71b-411b-9129-5b6431e4bbc9 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:47.416589 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-a3edff42-b71b-411b-9129-5b6431e4bbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:47.477514 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1331/2659] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:47.487017 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d0577-4a56-4116-b4fd-17186d2dc406 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:47.487332 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d0577-4a56-4116-b4fd-17186d2dc406 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:47.487650 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d0577-4a56-4116-b4fd-17186d2dc406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:47.508445 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-d22d0577-4a56-4116-b4fd-17186d2dc406 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:47.509575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d22d0577-4a56-4116-b4fd-17186d2dc406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:47.527358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1329/2660] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:47.528490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edff42-b71b-411b-9129-5b6431e4bbc9 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:47.538358 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edff42-b71b-411b-9129-5b6431e4bbc9 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:47.543037 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3751e2-9853-451f-adce-47a02153c9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:47.543767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3751e2-9853-451f-adce-47a02153c9df admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:47.544018 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3751e2-9853-451f-adce-47a02153c9df admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:47.544203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3751e2-9853-451f-adce-47a02153c9df admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:47.544846 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3751e2-9853-451f-adce-47a02153c9df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:47.545222 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3751e2-9853-451f-adce-47a02153c9df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:47.547562 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3751e2-9853-451f-adce-47a02153c9df admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:47.559336 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1332/2661] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:47.564739 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d0577-4a56-4116-b4fd-17186d2dc406 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:47.576310 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d0577-4a56-4116-b4fd-17186d2dc406 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:47.581532 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3244b-3a27-4331-b855-1b4fedf58fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:47.582136 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3244b-3a27-4331-b855-1b4fedf58fb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:47.584794 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3244b-3a27-4331-b855-1b4fedf58fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:47.584794 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3244b-3a27-4331-b855-1b4fedf58fb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:47.584794 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3244b-3a27-4331-b855-1b4fedf58fb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:47.585214 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3244b-3a27-4331-b855-1b4fedf58fb0 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:47.706507 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1330/2662] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:47 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:47.708235 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3751e2-9853-451f-adce-47a02153c9df admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:47.724674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3751e2-9853-451f-adce-47a02153c9df admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:47.798482 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74901854-ce27-47c9-9daa-683a31d8a065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:47.799054 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74901854-ce27-47c9-9daa-683a31d8a065 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:47.799335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74901854-ce27-47c9-9daa-683a31d8a065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:47.799607 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74901854-ce27-47c9-9daa-683a31d8a065 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:47.800246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74901854-ce27-47c9-9daa-683a31d8a065 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:47.802405 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74901854-ce27-47c9-9daa-683a31d8a065 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:47.812012 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:47.894924 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1333/2663] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:47 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:47.963294 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3244b-3a27-4331-b855-1b4fedf58fb0 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:47.970478 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:47.976221 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3244b-3a27-4331-b855-1b4fedf58fb0 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:47.985682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61815e5-2ba6-41d5-8ba3-25dc5bb99220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:47.986649 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61815e5-2ba6-41d5-8ba3-25dc5bb99220 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:47.987139 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61815e5-2ba6-41d5-8ba3-25dc5bb99220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:47.990389 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61815e5-2ba6-41d5-8ba3-25dc5bb99220 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:47.995481 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61815e5-2ba6-41d5-8ba3-25dc5bb99220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:47.997281 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61815e5-2ba6-41d5-8ba3-25dc5bb99220 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:48.010477 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1331/2664] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:48.062590 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74901854-ce27-47c9-9daa-683a31d8a065 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:48.070264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74901854-ce27-47c9-9daa-683a31d8a065 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:48.075054 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ad146-fe06-4626-b89b-a33dc18c33cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:48.075726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ad146-fe06-4626-b89b-a33dc18c33cd admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:48.076003 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ad146-fe06-4626-b89b-a33dc18c33cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:48.076275 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ad146-fe06-4626-b89b-a33dc18c33cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:48.076927 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ad146-fe06-4626-b89b-a33dc18c33cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:48.081380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ad146-fe06-4626-b89b-a33dc18c33cd admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:48.092568 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1334/2665] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:48.099466 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61815e5-2ba6-41d5-8ba3-25dc5bb99220 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:48.111972 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61815e5-2ba6-41d5-8ba3-25dc5bb99220 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:48.118240 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b679421-976a-41a9-858a-8218fb01ca99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:48.118849 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b679421-976a-41a9-858a-8218fb01ca99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:48.119184 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b679421-976a-41a9-858a-8218fb01ca99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:48.119460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b679421-976a-41a9-858a-8218fb01ca99 admin admin] PATH_INFO: `/v3/projects/0625c35e0855464fb21ac021aa887f8e/users/8bc1583454be4483b4c881627f8da287/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:48.144580 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b679421-976a-41a9-858a-8218fb01ca99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0625c35e0855464fb21ac021aa887f8e, user_id=8bc1583454be4483b4c881627f8da287, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:48.147885 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b679421-976a-41a9-858a-8218fb01ca99 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:48.272068 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1332/2666] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:48 2022] PUT /identity/v3/projects/0625c35e0855464fb21ac021aa887f8e/users/8bc1583454be4483b4c881627f8da287/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:48.273955 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41deaa-3abe-4cf3-9e8c-9d494ab20dd9 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:48.274240 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41deaa-3abe-4cf3-9e8c-9d494ab20dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:48.274514 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41deaa-3abe-4cf3-9e8c-9d494ab20dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:48.323388 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-4d41deaa-3abe-4cf3-9e8c-9d494ab20dd9 None None] MFA Rules not processed for user `8bc1583454be4483b4c881627f8da287`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:48.328061 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-4d41deaa-3abe-4cf3-9e8c-9d494ab20dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:48.397775 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1335/2667] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:48.410625 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b679421-976a-41a9-858a-8218fb01ca99 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:48.412701 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-1b679421-976a-41a9-858a-8218fb01ca99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:48.480138 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b679421-976a-41a9-858a-8218fb01ca99 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:48.483752 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5622d5-f929-41e6-9d06-d6fa9a8907d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:48.485204 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5622d5-f929-41e6-9d06-d6fa9a8907d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:48.485344 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5622d5-f929-41e6-9d06-d6fa9a8907d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:48.485505 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5622d5-f929-41e6-9d06-d6fa9a8907d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:48.495038 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5622d5-f929-41e6-9d06-d6fa9a8907d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:48.496468 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5622d5-f929-41e6-9d06-d6fa9a8907d2 service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:48.505795 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1333/2668] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:48.530603 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41deaa-3abe-4cf3-9e8c-9d494ab20dd9 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:48.601974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41deaa-3abe-4cf3-9e8c-9d494ab20dd9 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:48.608957 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ffd77-200f-4ef3-9c81-c56849fec8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:48.608957 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ffd77-200f-4ef3-9c81-c56849fec8ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:48.608957 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ffd77-200f-4ef3-9c81-c56849fec8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:48.608957 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ffd77-200f-4ef3-9c81-c56849fec8ac admin admin] PATH_INFO: `/v3/users/8bc1583454be4483b4c881627f8da287` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:48.609861 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ffd77-200f-4ef3-9c81-c56849fec8ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bc1583454be4483b4c881627f8da287)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:48.611806 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ffd77-200f-4ef3-9c81-c56849fec8ac admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:48.799750 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-688ffd77-200f-4ef3-9c81-c56849fec8ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bc1583454be4483b4c881627f8da287'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:48.883050 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-688ffd77-200f-4ef3-9c81-c56849fec8ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bc1583454be4483b4c881627f8da287'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:48.883624 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-688ffd77-200f-4ef3-9c81-c56849fec8ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bc1583454be4483b4c881627f8da287'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:48.893307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-688ffd77-200f-4ef3-9c81-c56849fec8ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bc1583454be4483b4c881627f8da287'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:48.905348 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-688ffd77-200f-4ef3-9c81-c56849fec8ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bc1583454be4483b4c881627f8da287'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:48.912348 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1336/2669] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:48 2022] DELETE /identity/v3/users/8bc1583454be4483b4c881627f8da287 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:48.916872 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccee8a-b404-440c-bc85-a25c016bd15f None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:48.917185 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccee8a-b404-440c-bc85-a25c016bd15f None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:48.917460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccee8a-b404-440c-bc85-a25c016bd15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:48.937686 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-1fccee8a-b404-440c-bc85-a25c016bd15f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:48.938993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-1fccee8a-b404-440c-bc85-a25c016bd15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:48.999790 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1334/2670] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:49.009854 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ffd77-200f-4ef3-9c81-c56849fec8ac admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:49.011322 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-688ffd77-200f-4ef3-9c81-c56849fec8ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:49.072794 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ffd77-200f-4ef3-9c81-c56849fec8ac admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:49.076457 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251c82c-e880-406f-b728-8de157ecdbdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:49.077063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251c82c-e880-406f-b728-8de157ecdbdc service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:49.077335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251c82c-e880-406f-b728-8de157ecdbdc service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:49.078154 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251c82c-e880-406f-b728-8de157ecdbdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:49.089633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251c82c-e880-406f-b728-8de157ecdbdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:49.091130 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251c82c-e880-406f-b728-8de157ecdbdc service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:49.101822 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1337/2671] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:49.366071 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:49.701486 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccee8a-b404-440c-bc85-a25c016bd15f None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:49.703956 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-1fccee8a-b404-440c-bc85-a25c016bd15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:49.716657 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccee8a-b404-440c-bc85-a25c016bd15f None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:49.722300 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd35e6-2a9c-4857-ac76-5dc4b45a4c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:49.726894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd35e6-2a9c-4857-ac76-5dc4b45a4c73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:49.726894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd35e6-2a9c-4857-ac76-5dc4b45a4c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:49.726894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd35e6-2a9c-4857-ac76-5dc4b45a4c73 admin admin] PATH_INFO: `/v3/projects/0625c35e0855464fb21ac021aa887f8e` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:49.726894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd35e6-2a9c-4857-ac76-5dc4b45a4c73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0625c35e0855464fb21ac021aa887f8e)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:49.726894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd35e6-2a9c-4857-ac76-5dc4b45a4c73 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:49.865058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-88bd35e6-2a9c-4857-ac76-5dc4b45a4c73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0625c35e0855464fb21ac021aa887f8e', 'request_id': 'req-88bd35e6-2a9c-4857-ac76-5dc4b45a4c73'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:49.870586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-88bd35e6-2a9c-4857-ac76-5dc4b45a4c73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0625c35e0855464fb21ac021aa887f8e', 'request_id': 'req-88bd35e6-2a9c-4857-ac76-5dc4b45a4c73'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:49.881676 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-88bd35e6-2a9c-4857-ac76-5dc4b45a4c73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0625c35e0855464fb21ac021aa887f8e', 'request_id': 'req-88bd35e6-2a9c-4857-ac76-5dc4b45a4c73'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:49.885789 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:49.959191 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-88bd35e6-2a9c-4857-ac76-5dc4b45a4c73 admin admin] The token cache is being invalidate because project 0625c35e0855464fb21ac021aa887f8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:49.959492 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-88bd35e6-2a9c-4857-ac76-5dc4b45a4c73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:49.960288 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-88bd35e6-2a9c-4857-ac76-5dc4b45a4c73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:49.962025 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1335/2672] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:49 2022] DELETE /identity/v3/projects/0625c35e0855464fb21ac021aa887f8e => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:49.966918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa689d-37e4-4e31-8142-a3754f56220a None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:49.968634 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa689d-37e4-4e31-8142-a3754f56220a None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:49.969950 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa689d-37e4-4e31-8142-a3754f56220a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:49.996117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-05fa689d-37e4-4e31-8142-a3754f56220a None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:49.997672 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-05fa689d-37e4-4e31-8142-a3754f56220a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:50.061699 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1338/2673] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:50.069358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7d417-1243-47dc-a299-39f790959fd0 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.070356 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7d417-1243-47dc-a299-39f790959fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.070487 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16ef62-91b1-4f91-9124-8c6d4c9394a7 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.070657 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7d417-1243-47dc-a299-39f790959fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.070772 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16ef62-91b1-4f91-9124-8c6d4c9394a7 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.070988 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16ef62-91b1-4f91-9124-8c6d4c9394a7 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.072428 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1339/2674] 10.222.0.13 () {54 vars in 921 bytes} [Mon Aug 8 17:01:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:01:50.079165 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc996229-3e7b-4bc8-9f84-b5aaa0b8b046 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.079780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc996229-3e7b-4bc8-9f84-b5aaa0b8b046 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.080606 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc996229-3e7b-4bc8-9f84-b5aaa0b8b046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.102975 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-e6c7d417-1243-47dc-a299-39f790959fd0 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:50.103830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-cc996229-3e7b-4bc8-9f84-b5aaa0b8b046 None None] MFA Rules not processed for user `9482bf89c1b648c1a91ee00ee5fbba63`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:50.105138 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-cc996229-3e7b-4bc8-9f84-b5aaa0b8b046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:50.105288 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-e6c7d417-1243-47dc-a299-39f790959fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:50.124004 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1336/2675] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:50.136464 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7d417-1243-47dc-a299-39f790959fd0 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:50.156317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7d417-1243-47dc-a299-39f790959fd0 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:50.156317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d3288-d806-4aaa-9bd5-6ada4f10c7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:50.156317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d3288-d806-4aaa-9bd5-6ada4f10c7c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.156317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d3288-d806-4aaa-9bd5-6ada4f10c7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.157483 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d3288-d806-4aaa-9bd5-6ada4f10c7c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.157483 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d3288-d806-4aaa-9bd5-6ada4f10c7c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:50.157483 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d3288-d806-4aaa-9bd5-6ada4f10c7c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:50.157827 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d3288-d806-4aaa-9bd5-6ada4f10c7c0 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:50.170435 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1340/2676] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Aug 8 17:01:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:50.172299 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1337/2677] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:01:50.176600 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc996229-3e7b-4bc8-9f84-b5aaa0b8b046 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:50.183820 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d3288-d806-4aaa-9bd5-6ada4f10c7c0 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:50.185924 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-3e9d3288-d806-4aaa-9bd5-6ada4f10c7c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:50.190743 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc996229-3e7b-4bc8-9f84-b5aaa0b8b046 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:50.195364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722aa15-554f-4682-9315-e91631af2e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:50.195970 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722aa15-554f-4682-9315-e91631af2e6b admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.196241 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722aa15-554f-4682-9315-e91631af2e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.196506 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722aa15-554f-4682-9315-e91631af2e6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.197134 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722aa15-554f-4682-9315-e91631af2e6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:50.198821 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d3288-d806-4aaa-9bd5-6ada4f10c7c0 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:50.199043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722aa15-554f-4682-9315-e91631af2e6b admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:50.202966 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8a152-0f19-4db8-9e56-159d670252e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:50.203625 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8a152-0f19-4db8-9e56-159d670252e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.203813 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8a152-0f19-4db8-9e56-159d670252e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.204094 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8a152-0f19-4db8-9e56-159d670252e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.212021 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8a152-0f19-4db8-9e56-159d670252e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:50.213842 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8a152-0f19-4db8-9e56-159d670252e8 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:50.223826 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1338/2678] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:50.290046 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1341/2679] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:50 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:50.292297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8a152-0f19-4db8-9e56-159d670252e8 service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:50.304618 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8a152-0f19-4db8-9e56-159d670252e8 service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:50.321025 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b76ee3-252b-4fb8-ad9d-af7f7bacb375 None None] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.321665 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b76ee3-252b-4fb8-ad9d-af7f7bacb375 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.322031 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b76ee3-252b-4fb8-ad9d-af7f7bacb375 None None] PATH_INFO: `/` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.323706 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1342/2680] 10.222.0.13 () {54 vars in 921 bytes} [Mon Aug 8 17:01:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:01:50.329935 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6efa9f-f59a-4d08-8270-b322bb6785c5 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.329935 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6efa9f-f59a-4d08-8270-b322bb6785c5 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.330349 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6efa9f-f59a-4d08-8270-b322bb6785c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.368382 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:50.372202 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-dd6efa9f-f59a-4d08-8270-b322bb6785c5 None None] MFA Rules not processed for user `9daa7489bced467984d39e8b8ac49d13`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:50.373433 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-dd6efa9f-f59a-4d08-8270-b322bb6785c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:50.381199 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc453b-21ad-46bf-8488-d851b1c5a261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:50.382069 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc453b-21ad-46bf-8488-d851b1c5a261 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.382350 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc453b-21ad-46bf-8488-d851b1c5a261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.382648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc453b-21ad-46bf-8488-d851b1c5a261 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.383278 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bc453b-21ad-46bf-8488-d851b1c5a261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:50.385351 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bc453b-21ad-46bf-8488-d851b1c5a261 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:50.455894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1343/2681] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Aug 8 17:01:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:50.467259 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6efa9f-f59a-4d08-8270-b322bb6785c5 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:50.468838 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-dd6efa9f-f59a-4d08-8270-b322bb6785c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:50.472933 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1339/2682] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:50 2022] POST /identity/v3/users => generated 288 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:50.480856 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc453b-21ad-46bf-8488-d851b1c5a261 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:50.492284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc453b-21ad-46bf-8488-d851b1c5a261 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:50.496981 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1feebf-c4ba-4224-bfd8-8b57072a8576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:50.506635 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1feebf-c4ba-4224-bfd8-8b57072a8576 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.506635 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1feebf-c4ba-4224-bfd8-8b57072a8576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.506635 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1feebf-c4ba-4224-bfd8-8b57072a8576 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.506635 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1feebf-c4ba-4224-bfd8-8b57072a8576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:50.506635 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1feebf-c4ba-4224-bfd8-8b57072a8576 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:50.513066 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1340/2683] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:50.517355 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1feebf-c4ba-4224-bfd8-8b57072a8576 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:50.527836 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1feebf-c4ba-4224-bfd8-8b57072a8576 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:50.532918 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38daa88e-4bd7-46e0-8512-f02be9982156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:50.533514 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38daa88e-4bd7-46e0-8512-f02be9982156 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.533849 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38daa88e-4bd7-46e0-8512-f02be9982156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.534188 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38daa88e-4bd7-46e0-8512-f02be9982156 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.534856 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38daa88e-4bd7-46e0-8512-f02be9982156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:50.536455 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38daa88e-4bd7-46e0-8512-f02be9982156 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:50.536578 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6efa9f-f59a-4d08-8270-b322bb6785c5 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:50.540293 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3061eb-ebd9-454c-925b-8f963de01eb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:50.540902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3061eb-ebd9-454c-925b-8f963de01eb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.541176 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3061eb-ebd9-454c-925b-8f963de01eb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.541447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3061eb-ebd9-454c-925b-8f963de01eb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.548085 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1341/2684] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:50.551893 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38daa88e-4bd7-46e0-8512-f02be9982156 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:50.555638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3061eb-ebd9-454c-925b-8f963de01eb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:50.557076 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3061eb-ebd9-454c-925b-8f963de01eb3 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:50.567001 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38daa88e-4bd7-46e0-8512-f02be9982156 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:50.570794 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1344/2685] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:50.577082 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c761c-ef39-4007-be18-b973271fcc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:50.577961 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c761c-ef39-4007-be18-b973271fcc63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.578282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c761c-ef39-4007-be18-b973271fcc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.578651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c761c-ef39-4007-be18-b973271fcc63 admin admin] PATH_INFO: `/v3/projects/4fb2e613a7774f0291ba9cccc18cfb31/users/eda9c43329f54e19bead646695ba29d2/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.598496 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c761c-ef39-4007-be18-b973271fcc63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fb2e613a7774f0291ba9cccc18cfb31, user_id=eda9c43329f54e19bead646695ba29d2, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:50.601892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c761c-ef39-4007-be18-b973271fcc63 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:50.705088 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1342/2686] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:50 2022] PUT /identity/v3/projects/4fb2e613a7774f0291ba9cccc18cfb31/users/eda9c43329f54e19bead646695ba29d2/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:50.716696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8126e41-c453-4895-ab44-e95699738439 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.716916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8126e41-c453-4895-ab44-e95699738439 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.717187 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8126e41-c453-4895-ab44-e95699738439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.752390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-b8126e41-c453-4895-ab44-e95699738439 None None] MFA Rules not processed for user `eda9c43329f54e19bead646695ba29d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:50.753905 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b8126e41-c453-4895-ab44-e95699738439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:50.823959 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1345/2687] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:50.834845 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c761c-ef39-4007-be18-b973271fcc63 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:50.836644 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-a21c761c-ef39-4007-be18-b973271fcc63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:50.900946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c761c-ef39-4007-be18-b973271fcc63 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:50.904889 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b99d2-e299-44dd-8d82-4873616a461c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:50.905487 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b99d2-e299-44dd-8d82-4873616a461c service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.906439 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b99d2-e299-44dd-8d82-4873616a461c service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.906785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b99d2-e299-44dd-8d82-4873616a461c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.915524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6b99d2-e299-44dd-8d82-4873616a461c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:50.916946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6b99d2-e299-44dd-8d82-4873616a461c service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:50.929851 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1343/2688] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:51.062012 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8126e41-c453-4895-ab44-e95699738439 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:51.063662 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b8126e41-c453-4895-ab44-e95699738439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:51.099301 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=100 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:51.129596 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8126e41-c453-4895-ab44-e95699738439 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:51.134280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1bec9-7a3e-43ba-8237-b130e082eea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:51.137678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1bec9-7a3e-43ba-8237-b130e082eea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:51.137678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1bec9-7a3e-43ba-8237-b130e082eea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:51.137678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1bec9-7a3e-43ba-8237-b130e082eea8 admin admin] PATH_INFO: `/v3/users/a4e78c3d7a1c4ee49c3755298f132951` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:51.137678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b1bec9-7a3e-43ba-8237-b130e082eea8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4e78c3d7a1c4ee49c3755298f132951)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:51.138500 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b1bec9-7a3e-43ba-8237-b130e082eea8 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:51.386479 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-11b1bec9-7a3e-43ba-8237-b130e082eea8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4e78c3d7a1c4ee49c3755298f132951'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:51.469740 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-11b1bec9-7a3e-43ba-8237-b130e082eea8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4e78c3d7a1c4ee49c3755298f132951'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:51.470390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-11b1bec9-7a3e-43ba-8237-b130e082eea8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4e78c3d7a1c4ee49c3755298f132951'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:51.479816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-11b1bec9-7a3e-43ba-8237-b130e082eea8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4e78c3d7a1c4ee49c3755298f132951'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:51.492921 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-11b1bec9-7a3e-43ba-8237-b130e082eea8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4e78c3d7a1c4ee49c3755298f132951'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:51.503246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1346/2689] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:51 2022] DELETE /identity/v3/users/a4e78c3d7a1c4ee49c3755298f132951 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:51.509564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27673ea8-49d1-4710-b107-d2e1cf89f3a8 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:51.510055 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27673ea8-49d1-4710-b107-d2e1cf89f3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:51.510533 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27673ea8-49d1-4710-b107-d2e1cf89f3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:51.526339 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:51.533124 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-27673ea8-49d1-4710-b107-d2e1cf89f3a8 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:51.534330 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-27673ea8-49d1-4710-b107-d2e1cf89f3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:51.599668 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1344/2690] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:51.609158 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1bec9-7a3e-43ba-8237-b130e082eea8 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:51.610629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-11b1bec9-7a3e-43ba-8237-b130e082eea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:51.684454 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1bec9-7a3e-43ba-8237-b130e082eea8 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:51.689433 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7615c5-fe38-41b2-92e4-cc82925f8ce0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:51.690728 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7615c5-fe38-41b2-92e4-cc82925f8ce0 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:51.690728 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7615c5-fe38-41b2-92e4-cc82925f8ce0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:51.690728 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7615c5-fe38-41b2-92e4-cc82925f8ce0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:51.704781 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7615c5-fe38-41b2-92e4-cc82925f8ce0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:51.705198 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7615c5-fe38-41b2-92e4-cc82925f8ce0 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:51.719638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1347/2691] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:52.227892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27673ea8-49d1-4710-b107-d2e1cf89f3a8 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:52.229516 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-27673ea8-49d1-4710-b107-d2e1cf89f3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:52.243738 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27673ea8-49d1-4710-b107-d2e1cf89f3a8 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:52.248978 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ceacc3-76f5-4aeb-936b-ed49b4b83868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:52.249860 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ceacc3-76f5-4aeb-936b-ed49b4b83868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:52.250287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ceacc3-76f5-4aeb-936b-ed49b4b83868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:52.250786 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ceacc3-76f5-4aeb-936b-ed49b4b83868 admin admin] PATH_INFO: `/v3/projects/c8c176f660e8470eb128102c1f7aa8d0` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:52.252172 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ceacc3-76f5-4aeb-936b-ed49b4b83868 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8c176f660e8470eb128102c1f7aa8d0)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:52.254371 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ceacc3-76f5-4aeb-936b-ed49b4b83868 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:52.434055 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-25ceacc3-76f5-4aeb-936b-ed49b4b83868 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8c176f660e8470eb128102c1f7aa8d0', 'request_id': 'req-25ceacc3-76f5-4aeb-936b-ed49b4b83868'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:52.440972 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-25ceacc3-76f5-4aeb-936b-ed49b4b83868 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8c176f660e8470eb128102c1f7aa8d0', 'request_id': 'req-25ceacc3-76f5-4aeb-936b-ed49b4b83868'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:52.449058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-25ceacc3-76f5-4aeb-936b-ed49b4b83868 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8c176f660e8470eb128102c1f7aa8d0', 'request_id': 'req-25ceacc3-76f5-4aeb-936b-ed49b4b83868'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:52.600902 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-25ceacc3-76f5-4aeb-936b-ed49b4b83868 admin admin] The token cache is being invalidate because project c8c176f660e8470eb128102c1f7aa8d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:52.602131 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-25ceacc3-76f5-4aeb-936b-ed49b4b83868 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:52.602457 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-25ceacc3-76f5-4aeb-936b-ed49b4b83868 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:52.604489 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1345/2692] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:52 2022] DELETE /identity/v3/projects/c8c176f660e8470eb128102c1f7aa8d0 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:52.609251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7615c5-fe38-41b2-92e4-cc82925f8ce0 service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:52.622709 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7615c5-fe38-41b2-92e4-cc82925f8ce0 service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:52.702110 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:52.714147 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf070e-e13e-4e21-a7dc-68f1d1f4f01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:52.714837 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf070e-e13e-4e21-a7dc-68f1d1f4f01f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:52.715118 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf070e-e13e-4e21-a7dc-68f1d1f4f01f admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:52.715411 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf070e-e13e-4e21-a7dc-68f1d1f4f01f admin admin] PATH_INFO: `/v3/users/4eb7018c3c3142d784f0d49b3bfa151f` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:52.716584 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf070e-e13e-4e21-a7dc-68f1d1f4f01f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eb7018c3c3142d784f0d49b3bfa151f)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:52.719330 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf070e-e13e-4e21-a7dc-68f1d1f4f01f admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:52.829421 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9fe9c-3c12-4660-8129-015fbaa7bb23 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:52.829694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9fe9c-3c12-4660-8129-015fbaa7bb23 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:52.829960 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9fe9c-3c12-4660-8129-015fbaa7bb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:52.858822 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-68c9fe9c-3c12-4660-8129-015fbaa7bb23 None None] MFA Rules not processed for user `eda9c43329f54e19bead646695ba29d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:52.860286 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-68c9fe9c-3c12-4660-8129-015fbaa7bb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:52.925623 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1346/2693] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:52.969807 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0faf070e-e13e-4e21-a7dc-68f1d1f4f01f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eb7018c3c3142d784f0d49b3bfa151f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:53.106335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0faf070e-e13e-4e21-a7dc-68f1d1f4f01f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eb7018c3c3142d784f0d49b3bfa151f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:53.107927 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0faf070e-e13e-4e21-a7dc-68f1d1f4f01f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eb7018c3c3142d784f0d49b3bfa151f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:53.118790 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0faf070e-e13e-4e21-a7dc-68f1d1f4f01f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eb7018c3c3142d784f0d49b3bfa151f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:53.133709 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0faf070e-e13e-4e21-a7dc-68f1d1f4f01f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eb7018c3c3142d784f0d49b3bfa151f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:53.144780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1348/2694] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:52 2022] DELETE /identity/v3/users/4eb7018c3c3142d784f0d49b3bfa151f => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:53.715147 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9fe9c-3c12-4660-8129-015fbaa7bb23 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:53.716679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-68c9fe9c-3c12-4660-8129-015fbaa7bb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:53.750053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9fe9c-3c12-4660-8129-015fbaa7bb23 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:53.820216 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:53.841829 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454882c1-fa85-4411-a24d-013bc23315e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:53.842473 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454882c1-fa85-4411-a24d-013bc23315e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:53.842782 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454882c1-fa85-4411-a24d-013bc23315e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:53.843584 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454882c1-fa85-4411-a24d-013bc23315e1 admin admin] PATH_INFO: `/v3/projects/948bf3d813dd493abda63120d53952d5` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:53.849028 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454882c1-fa85-4411-a24d-013bc23315e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=948bf3d813dd493abda63120d53952d5)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:53.849028 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454882c1-fa85-4411-a24d-013bc23315e1 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:54.021343 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-454882c1-fa85-4411-a24d-013bc23315e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '948bf3d813dd493abda63120d53952d5', 'request_id': 'req-454882c1-fa85-4411-a24d-013bc23315e1'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:54.027532 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-454882c1-fa85-4411-a24d-013bc23315e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '948bf3d813dd493abda63120d53952d5', 'request_id': 'req-454882c1-fa85-4411-a24d-013bc23315e1'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:54.037101 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-454882c1-fa85-4411-a24d-013bc23315e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '948bf3d813dd493abda63120d53952d5', 'request_id': 'req-454882c1-fa85-4411-a24d-013bc23315e1'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:54.145602 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-454882c1-fa85-4411-a24d-013bc23315e1 admin admin] The token cache is being invalidate because project 948bf3d813dd493abda63120d53952d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:54.147818 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-454882c1-fa85-4411-a24d-013bc23315e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:54.147818 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-454882c1-fa85-4411-a24d-013bc23315e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:54.148473 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1347/2695] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:53 2022] DELETE /identity/v3/projects/948bf3d813dd493abda63120d53952d5 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:54.155114 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e94c5-fdb8-4f85-9e70-0b793062ea93 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:54.155731 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e94c5-fdb8-4f85-9e70-0b793062ea93 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:54.155731 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e94c5-fdb8-4f85-9e70-0b793062ea93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:54.178583 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-875e94c5-fdb8-4f85-9e70-0b793062ea93 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:54.179908 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-875e94c5-fdb8-4f85-9e70-0b793062ea93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:54.256942 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1349/2696] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:54.258990 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386640b-6524-4484-8124-966e66bb91eb None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:54.269579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386640b-6524-4484-8124-966e66bb91eb None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:54.269579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386640b-6524-4484-8124-966e66bb91eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:54.282227 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-0386640b-6524-4484-8124-966e66bb91eb None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:54.283636 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-0386640b-6524-4484-8124-966e66bb91eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:54.295426 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1348/2697] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:54.300469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e94c5-fdb8-4f85-9e70-0b793062ea93 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:54.312483 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e94c5-fdb8-4f85-9e70-0b793062ea93 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:54.319054 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ad12d-7f19-48b4-8e2c-0d6fa21b6073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:54.320083 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ad12d-7f19-48b4-8e2c-0d6fa21b6073 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:54.320416 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ad12d-7f19-48b4-8e2c-0d6fa21b6073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:54.320691 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ad12d-7f19-48b4-8e2c-0d6fa21b6073 admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:54.321487 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ad12d-7f19-48b4-8e2c-0d6fa21b6073 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:54.321759 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ad12d-7f19-48b4-8e2c-0d6fa21b6073 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:54.323404 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ad12d-7f19-48b4-8e2c-0d6fa21b6073 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:54.339287 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1350/2698] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:01:54.344474 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386640b-6524-4484-8124-966e66bb91eb None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:54.355053 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386640b-6524-4484-8124-966e66bb91eb None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:54.359576 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597e6ba-a76d-4249-bfa1-3f769b1d999c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:54.360220 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597e6ba-a76d-4249-bfa1-3f769b1d999c admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:54.360491 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597e6ba-a76d-4249-bfa1-3f769b1d999c admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:54.360752 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597e6ba-a76d-4249-bfa1-3f769b1d999c admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:54.361385 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597e6ba-a76d-4249-bfa1-3f769b1d999c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:54.363327 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597e6ba-a76d-4249-bfa1-3f769b1d999c admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:54.541815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1349/2699] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:54 2022] POST /identity/v3/projects => generated 389 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:54.546087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ad12d-7f19-48b4-8e2c-0d6fa21b6073 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:54.557526 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ad12d-7f19-48b4-8e2c-0d6fa21b6073 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:54.625437 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13cbb6-6231-4cd2-9fa5-d91c4b701b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:54.626375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13cbb6-6231-4cd2-9fa5-d91c4b701b04 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:54.626534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13cbb6-6231-4cd2-9fa5-d91c4b701b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:54.626867 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13cbb6-6231-4cd2-9fa5-d91c4b701b04 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:54.628020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13cbb6-6231-4cd2-9fa5-d91c4b701b04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:54.629989 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13cbb6-6231-4cd2-9fa5-d91c4b701b04 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:54.638861 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:54.799478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1351/2700] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:54 2022] POST /identity/v3/users => generated 295 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:54.804577 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597e6ba-a76d-4249-bfa1-3f769b1d999c admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:54.823412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597e6ba-a76d-4249-bfa1-3f769b1d999c admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:54.832270 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8a9a7-7168-476a-af22-e1229c7b9721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:54.833362 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8a9a7-7168-476a-af22-e1229c7b9721 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:54.833871 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8a9a7-7168-476a-af22-e1229c7b9721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:54.834338 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8a9a7-7168-476a-af22-e1229c7b9721 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:54.835643 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8a9a7-7168-476a-af22-e1229c7b9721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:54.838881 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8a9a7-7168-476a-af22-e1229c7b9721 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:54.852524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1350/2701] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:54.857278 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13cbb6-6231-4cd2-9fa5-d91c4b701b04 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:54.868492 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13cbb6-6231-4cd2-9fa5-d91c4b701b04 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:54.873319 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeeafe1-0390-46c6-9bed-5afd43bc818a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:54.873841 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeeafe1-0390-46c6-9bed-5afd43bc818a admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:54.874149 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeeafe1-0390-46c6-9bed-5afd43bc818a admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:54.874456 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeeafe1-0390-46c6-9bed-5afd43bc818a admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:54.875750 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeeafe1-0390-46c6-9bed-5afd43bc818a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:54.877331 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeeafe1-0390-46c6-9bed-5afd43bc818a admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:54.889046 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1352/2702] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:54.898147 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8a9a7-7168-476a-af22-e1229c7b9721 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:54.909230 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8a9a7-7168-476a-af22-e1229c7b9721 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:54.914230 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c61567-32e8-4b42-8ae3-5460ae11156b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:54.914941 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c61567-32e8-4b42-8ae3-5460ae11156b admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:54.915264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c61567-32e8-4b42-8ae3-5460ae11156b admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:54.915534 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c61567-32e8-4b42-8ae3-5460ae11156b admin admin] PATH_INFO: `/v3/projects/6018b22d0d4c4999931ce8f1fc224c27/users/5cd87823e46743f5b242eca1faa3ad77/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:54.940809 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c61567-32e8-4b42-8ae3-5460ae11156b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6018b22d0d4c4999931ce8f1fc224c27, user_id=5cd87823e46743f5b242eca1faa3ad77, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:54.947187 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c61567-32e8-4b42-8ae3-5460ae11156b admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:55.062508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1351/2703] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:54 2022] PUT /identity/v3/projects/6018b22d0d4c4999931ce8f1fc224c27/users/5cd87823e46743f5b242eca1faa3ad77/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:55.080598 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04173f48-a846-48e1-90da-9d68fda79bbf None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:55.080900 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04173f48-a846-48e1-90da-9d68fda79bbf None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:55.081129 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04173f48-a846-48e1-90da-9d68fda79bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:55.116633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-04173f48-a846-48e1-90da-9d68fda79bbf None None] MFA Rules not processed for user `5cd87823e46743f5b242eca1faa3ad77`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:55.117908 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-04173f48-a846-48e1-90da-9d68fda79bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:55.192735 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1353/2704] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:55 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:55.205197 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c61567-32e8-4b42-8ae3-5460ae11156b admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:55.207277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-32c61567-32e8-4b42-8ae3-5460ae11156b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:55.273463 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c61567-32e8-4b42-8ae3-5460ae11156b admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:55.277143 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68044f6-c054-4ff0-8a2a-7530329e030a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:55.277749 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68044f6-c054-4ff0-8a2a-7530329e030a service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:55.278056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68044f6-c054-4ff0-8a2a-7530329e030a service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:55.278318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68044f6-c054-4ff0-8a2a-7530329e030a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:55.288825 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68044f6-c054-4ff0-8a2a-7530329e030a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:55.290324 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68044f6-c054-4ff0-8a2a-7530329e030a service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:55.306362 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1352/2705] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:55 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:57.968970 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04173f48-a846-48e1-90da-9d68fda79bbf None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:57.970506 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-04173f48-a846-48e1-90da-9d68fda79bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:58.051965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04173f48-a846-48e1-90da-9d68fda79bbf None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:58.057002 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac4432-80a8-4ebe-b3d1-4a9cc6c1d3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:58.057672 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac4432-80a8-4ebe-b3d1-4a9cc6c1d3ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:58.057954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac4432-80a8-4ebe-b3d1-4a9cc6c1d3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:58.058231 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac4432-80a8-4ebe-b3d1-4a9cc6c1d3ac admin admin] PATH_INFO: `/v3/users/b15ebc8995444db3a2a0dd89fe9bc6c4` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:58.059434 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac4432-80a8-4ebe-b3d1-4a9cc6c1d3ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=b15ebc8995444db3a2a0dd89fe9bc6c4)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:58.061761 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac4432-80a8-4ebe-b3d1-4a9cc6c1d3ac admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:58.241506 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-50ac4432-80a8-4ebe-b3d1-4a9cc6c1d3ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b15ebc8995444db3a2a0dd89fe9bc6c4'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:58.321534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-50ac4432-80a8-4ebe-b3d1-4a9cc6c1d3ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b15ebc8995444db3a2a0dd89fe9bc6c4'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:58.323031 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-50ac4432-80a8-4ebe-b3d1-4a9cc6c1d3ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b15ebc8995444db3a2a0dd89fe9bc6c4'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:58.332357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-50ac4432-80a8-4ebe-b3d1-4a9cc6c1d3ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b15ebc8995444db3a2a0dd89fe9bc6c4'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:58.338538 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:58.352728 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-50ac4432-80a8-4ebe-b3d1-4a9cc6c1d3ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b15ebc8995444db3a2a0dd89fe9bc6c4'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:58.360138 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1354/2706] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:57 2022] DELETE /identity/v3/users/b15ebc8995444db3a2a0dd89fe9bc6c4 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:58.367079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a861ffa-edf5-4eb0-bd3c-3ae4ee19378e None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:58.367663 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a861ffa-edf5-4eb0-bd3c-3ae4ee19378e None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:58.367663 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a861ffa-edf5-4eb0-bd3c-3ae4ee19378e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:58.376437 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:58.387527 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-2a861ffa-edf5-4eb0-bd3c-3ae4ee19378e None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:58.388750 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-2a861ffa-edf5-4eb0-bd3c-3ae4ee19378e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:58.453682 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1353/2707] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:58.468028 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac4432-80a8-4ebe-b3d1-4a9cc6c1d3ac admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:58.470897 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-50ac4432-80a8-4ebe-b3d1-4a9cc6c1d3ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:58.555586 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac4432-80a8-4ebe-b3d1-4a9cc6c1d3ac admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:58.562505 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7e7a4-a59d-4ba9-80a6-1bb036682dee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:58.563449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7e7a4-a59d-4ba9-80a6-1bb036682dee service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:58.563910 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7e7a4-a59d-4ba9-80a6-1bb036682dee service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:58.565007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7e7a4-a59d-4ba9-80a6-1bb036682dee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:58.578690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7e7a4-a59d-4ba9-80a6-1bb036682dee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:58.585499 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7e7a4-a59d-4ba9-80a6-1bb036682dee service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:58.591545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1355/2708] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:59.117870 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a861ffa-edf5-4eb0-bd3c-3ae4ee19378e None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:59.119390 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-2a861ffa-edf5-4eb0-bd3c-3ae4ee19378e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:59.133472 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a861ffa-edf5-4eb0-bd3c-3ae4ee19378e None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:59.139859 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c75d87-a96b-4164-9b8d-8ecb103fb8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:59.142041 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c75d87-a96b-4164-9b8d-8ecb103fb8ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:59.142041 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c75d87-a96b-4164-9b8d-8ecb103fb8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:59.142041 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c75d87-a96b-4164-9b8d-8ecb103fb8ed admin admin] PATH_INFO: `/v3/projects/c26bb29455ee4331ae95bce891d66bdc` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:59.143360 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c75d87-a96b-4164-9b8d-8ecb103fb8ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=c26bb29455ee4331ae95bce891d66bdc)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:59.145301 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c75d87-a96b-4164-9b8d-8ecb103fb8ed admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:59.273932 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e9c75d87-a96b-4164-9b8d-8ecb103fb8ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c26bb29455ee4331ae95bce891d66bdc', 'request_id': 'req-e9c75d87-a96b-4164-9b8d-8ecb103fb8ed'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:59.289796 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e9c75d87-a96b-4164-9b8d-8ecb103fb8ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c26bb29455ee4331ae95bce891d66bdc', 'request_id': 'req-e9c75d87-a96b-4164-9b8d-8ecb103fb8ed'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:59.299955 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e9c75d87-a96b-4164-9b8d-8ecb103fb8ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c26bb29455ee4331ae95bce891d66bdc', 'request_id': 'req-e9c75d87-a96b-4164-9b8d-8ecb103fb8ed'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:59.379527 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e9c75d87-a96b-4164-9b8d-8ecb103fb8ed admin admin] The token cache is being invalidate because project c26bb29455ee4331ae95bce891d66bdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:59.380279 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e9c75d87-a96b-4164-9b8d-8ecb103fb8ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:59.380680 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e9c75d87-a96b-4164-9b8d-8ecb103fb8ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:59.382467 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1354/2709] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:59 2022] DELETE /identity/v3/projects/c26bb29455ee4331ae95bce891d66bdc => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:59.388080 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05ba29-1328-44e1-927c-563b09d009e5 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:59.388894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05ba29-1328-44e1-927c-563b09d009e5 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:59.388894 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05ba29-1328-44e1-927c-563b09d009e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:59.412369 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-4e05ba29-1328-44e1-927c-563b09d009e5 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:59.413814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-4e05ba29-1328-44e1-927c-563b09d009e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:59.500977 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1356/2710] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:59.512544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b3f8e-e641-491f-b8df-c0c877a83f6b None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:59.513054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b3f8e-e641-491f-b8df-c0c877a83f6b None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:59.513526 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b3f8e-e641-491f-b8df-c0c877a83f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:59.545215 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-5d8b3f8e-e641-491f-b8df-c0c877a83f6b None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:59.547067 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-5d8b3f8e-e641-491f-b8df-c0c877a83f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:59.555498 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1355/2711] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:59.561588 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05ba29-1328-44e1-927c-563b09d009e5 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:59.572443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05ba29-1328-44e1-927c-563b09d009e5 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:59.577111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c455fb0-bf86-48b8-9750-3c3ac15e1ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:59.579918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c455fb0-bf86-48b8-9750-3c3ac15e1ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:59.579918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c455fb0-bf86-48b8-9750-3c3ac15e1ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:59.579918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c455fb0-bf86-48b8-9750-3c3ac15e1ab5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:59.579918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c455fb0-bf86-48b8-9750-3c3ac15e1ab5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:59.579918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c455fb0-bf86-48b8-9750-3c3ac15e1ab5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:59.580991 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c455fb0-bf86-48b8-9750-3c3ac15e1ab5 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:59.593880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1357/2712] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:59.598890 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b3f8e-e641-491f-b8df-c0c877a83f6b None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:59.609759 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b3f8e-e641-491f-b8df-c0c877a83f6b None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:59.615626 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb26a62-a4f4-4b87-a2ba-3441c454acd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:59.615626 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb26a62-a4f4-4b87-a2ba-3441c454acd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:59.615626 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb26a62-a4f4-4b87-a2ba-3441c454acd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:59.616255 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb26a62-a4f4-4b87-a2ba-3441c454acd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:59.616529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb26a62-a4f4-4b87-a2ba-3441c454acd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:59.618930 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb26a62-a4f4-4b87-a2ba-3441c454acd0 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:59.725012 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1356/2713] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:59 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:59.729705 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c455fb0-bf86-48b8-9750-3c3ac15e1ab5 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:59.741219 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c455fb0-bf86-48b8-9750-3c3ac15e1ab5 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:59.812999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab20d8-5181-4d6b-8eb0-2d305a6a1fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:59.813622 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab20d8-5181-4d6b-8eb0-2d305a6a1fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:59.813905 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab20d8-5181-4d6b-8eb0-2d305a6a1fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:59.814192 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab20d8-5181-4d6b-8eb0-2d305a6a1fc4 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:59.814848 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab20d8-5181-4d6b-8eb0-2d305a6a1fc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:59.816802 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab20d8-5181-4d6b-8eb0-2d305a6a1fc4 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:59.939659 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1358/2714] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:59 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:59.943862 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb26a62-a4f4-4b87-a2ba-3441c454acd0 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:59.954862 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb26a62-a4f4-4b87-a2ba-3441c454acd0 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:59.959461 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fafff8-2cd7-4da9-99ae-771bacb89377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:59.960069 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fafff8-2cd7-4da9-99ae-771bacb89377 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:59.960346 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fafff8-2cd7-4da9-99ae-771bacb89377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:59.960610 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fafff8-2cd7-4da9-99ae-771bacb89377 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:59.961254 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fafff8-2cd7-4da9-99ae-771bacb89377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:59.962927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fafff8-2cd7-4da9-99ae-771bacb89377 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:59.974359 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1357/2715] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:59.978492 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab20d8-5181-4d6b-8eb0-2d305a6a1fc4 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:59.992069 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab20d8-5181-4d6b-8eb0-2d305a6a1fc4 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:00.000838 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c841451-c0ca-4eed-acaa-777460dbe023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:00.001757 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c841451-c0ca-4eed-acaa-777460dbe023 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:00.002230 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c841451-c0ca-4eed-acaa-777460dbe023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:00.002930 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c841451-c0ca-4eed-acaa-777460dbe023 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:00.004091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c841451-c0ca-4eed-acaa-777460dbe023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:00.007242 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c841451-c0ca-4eed-acaa-777460dbe023 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:00.009714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:00.021307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1359/2716] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:02:00.027314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fafff8-2cd7-4da9-99ae-771bacb89377 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:00.037964 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fafff8-2cd7-4da9-99ae-771bacb89377 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:00.042758 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b2cd5-d381-4f78-bfad-393f5babb297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:00.043342 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b2cd5-d381-4f78-bfad-393f5babb297 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:00.043621 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b2cd5-d381-4f78-bfad-393f5babb297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:00.043894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b2cd5-d381-4f78-bfad-393f5babb297 admin admin] PATH_INFO: `/v3/projects/feff05ece7b6455097081d08ce2ef09e/users/73481d240f384622a2069274f9fc755a/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:00.062710 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b2cd5-d381-4f78-bfad-393f5babb297 admin admin] RBAC: Authorizing `identity:create_grant(project_id=feff05ece7b6455097081d08ce2ef09e, user_id=73481d240f384622a2069274f9fc755a, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:00.065364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b2cd5-d381-4f78-bfad-393f5babb297 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:00.168660 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1358/2717] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:02:00 2022] PUT /identity/v3/projects/feff05ece7b6455097081d08ce2ef09e/users/73481d240f384622a2069274f9fc755a/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:00.181270 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b025926-378e-4af2-ae01-b5d200dae7b2 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:00.181555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b025926-378e-4af2-ae01-b5d200dae7b2 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:00.181845 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b025926-378e-4af2-ae01-b5d200dae7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:00.215480 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-2b025926-378e-4af2-ae01-b5d200dae7b2 None None] MFA Rules not processed for user `73481d240f384622a2069274f9fc755a`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:00.216738 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-2b025926-378e-4af2-ae01-b5d200dae7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:00.291318 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1360/2718] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:02:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:00.303555 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b2cd5-d381-4f78-bfad-393f5babb297 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:00.305394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-b81b2cd5-d381-4f78-bfad-393f5babb297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:00.337705 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:00.371898 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b2cd5-d381-4f78-bfad-393f5babb297 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:00.376631 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655454f3-b5ef-4269-8f10-8cd792a1b59d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:00.377247 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655454f3-b5ef-4269-8f10-8cd792a1b59d service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:00.377520 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655454f3-b5ef-4269-8f10-8cd792a1b59d service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:00.377787 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655454f3-b5ef-4269-8f10-8cd792a1b59d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:00.386735 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655454f3-b5ef-4269-8f10-8cd792a1b59d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:00.388400 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655454f3-b5ef-4269-8f10-8cd792a1b59d service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:00.398321 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1359/2719] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:02:06.877781 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b025926-378e-4af2-ae01-b5d200dae7b2 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:07.016500 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b025926-378e-4af2-ae01-b5d200dae7b2 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:07.022029 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80f487-4904-4dde-85be-3ca86d1e6f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:07.023586 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80f487-4904-4dde-85be-3ca86d1e6f70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:07.023586 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80f487-4904-4dde-85be-3ca86d1e6f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:07.023586 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80f487-4904-4dde-85be-3ca86d1e6f70 admin admin] PATH_INFO: `/v3/users/73481d240f384622a2069274f9fc755a` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:07.027387 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f80f487-4904-4dde-85be-3ca86d1e6f70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73481d240f384622a2069274f9fc755a)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:07.029374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f80f487-4904-4dde-85be-3ca86d1e6f70 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:07.255862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-6f80f487-4904-4dde-85be-3ca86d1e6f70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73481d240f384622a2069274f9fc755a'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:07.351695 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-6f80f487-4904-4dde-85be-3ca86d1e6f70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73481d240f384622a2069274f9fc755a'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:07.352615 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-6f80f487-4904-4dde-85be-3ca86d1e6f70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73481d240f384622a2069274f9fc755a'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:07.367434 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-6f80f487-4904-4dde-85be-3ca86d1e6f70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73481d240f384622a2069274f9fc755a'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:07.389611 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-6f80f487-4904-4dde-85be-3ca86d1e6f70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73481d240f384622a2069274f9fc755a'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:07.401602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1361/2720] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:02:06 2022] DELETE /identity/v3/users/73481d240f384622a2069274f9fc755a => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:07.408257 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c507f-c868-4d86-a7a9-e975681a8443 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:07.408832 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c507f-c868-4d86-a7a9-e975681a8443 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:07.408832 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c507f-c868-4d86-a7a9-e975681a8443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:07.442263 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-7f8c507f-c868-4d86-a7a9-e975681a8443 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:07.444587 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-7f8c507f-c868-4d86-a7a9-e975681a8443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:07.536039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1360/2721] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:02:07.551413 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80f487-4904-4dde-85be-3ca86d1e6f70 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:07.553411 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-6f80f487-4904-4dde-85be-3ca86d1e6f70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:07.561804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:07.651953 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80f487-4904-4dde-85be-3ca86d1e6f70 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:07.658026 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27428f87-339c-4f25-97c3-66f16e483a1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:07.659244 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27428f87-339c-4f25-97c3-66f16e483a1b service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:07.659884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27428f87-339c-4f25-97c3-66f16e483a1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:07.660329 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27428f87-339c-4f25-97c3-66f16e483a1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:07.672410 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27428f87-339c-4f25-97c3-66f16e483a1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:07.674255 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27428f87-339c-4f25-97c3-66f16e483a1b service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:07.692441 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1362/2722] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:02:08.290914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c507f-c868-4d86-a7a9-e975681a8443 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:08.293132 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-7f8c507f-c868-4d86-a7a9-e975681a8443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:08.322455 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c507f-c868-4d86-a7a9-e975681a8443 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:08.332339 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f836d3-d623-4500-aa0e-5068b7f00c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:08.333415 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f836d3-d623-4500-aa0e-5068b7f00c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:08.333867 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f836d3-d623-4500-aa0e-5068b7f00c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:08.334334 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f836d3-d623-4500-aa0e-5068b7f00c96 admin admin] PATH_INFO: `/v3/projects/feff05ece7b6455097081d08ce2ef09e` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:08.335997 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f836d3-d623-4500-aa0e-5068b7f00c96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=feff05ece7b6455097081d08ce2ef09e)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:08.338401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f836d3-d623-4500-aa0e-5068b7f00c96 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:08.494782 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f7f836d3-d623-4500-aa0e-5068b7f00c96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feff05ece7b6455097081d08ce2ef09e', 'request_id': 'req-f7f836d3-d623-4500-aa0e-5068b7f00c96'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:08.511229 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f7f836d3-d623-4500-aa0e-5068b7f00c96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feff05ece7b6455097081d08ce2ef09e', 'request_id': 'req-f7f836d3-d623-4500-aa0e-5068b7f00c96'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:08.519685 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f7f836d3-d623-4500-aa0e-5068b7f00c96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feff05ece7b6455097081d08ce2ef09e', 'request_id': 'req-f7f836d3-d623-4500-aa0e-5068b7f00c96'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:08.619469 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f7f836d3-d623-4500-aa0e-5068b7f00c96 admin admin] The token cache is being invalidate because project feff05ece7b6455097081d08ce2ef09e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:02:08.619765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f7f836d3-d623-4500-aa0e-5068b7f00c96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:08.620460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f7f836d3-d623-4500-aa0e-5068b7f00c96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:08.622026 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1361/2723] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:02:08 2022] DELETE /identity/v3/projects/feff05ece7b6455097081d08ce2ef09e => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:02:17.009508 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27428f87-339c-4f25-97c3-66f16e483a1b service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:17.010954 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-27428f87-339c-4f25-97c3-66f16e483a1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:17.024408 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27428f87-339c-4f25-97c3-66f16e483a1b service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:17.083074 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45272e01-eebe-4e03-92b4-1529546b972a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:17.083725 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45272e01-eebe-4e03-92b4-1529546b972a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:17.084163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45272e01-eebe-4e03-92b4-1529546b972a admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:17.084459 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45272e01-eebe-4e03-92b4-1529546b972a admin admin] PATH_INFO: `/v3/users/5cd87823e46743f5b242eca1faa3ad77` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:17.085387 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45272e01-eebe-4e03-92b4-1529546b972a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cd87823e46743f5b242eca1faa3ad77)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:17.087321 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45272e01-eebe-4e03-92b4-1529546b972a admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:17.273939 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-45272e01-eebe-4e03-92b4-1529546b972a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cd87823e46743f5b242eca1faa3ad77'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:17.349104 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-45272e01-eebe-4e03-92b4-1529546b972a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cd87823e46743f5b242eca1faa3ad77'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:17.349717 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-45272e01-eebe-4e03-92b4-1529546b972a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cd87823e46743f5b242eca1faa3ad77'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:17.357420 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-45272e01-eebe-4e03-92b4-1529546b972a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cd87823e46743f5b242eca1faa3ad77'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:17.371109 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-45272e01-eebe-4e03-92b4-1529546b972a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cd87823e46743f5b242eca1faa3ad77'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:17.378511 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1363/2724] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:02:17 2022] DELETE /identity/v3/users/5cd87823e46743f5b242eca1faa3ad77 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:17.383061 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c3c31-39f1-4325-9926-35966af7e14c None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:17.383552 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c3c31-39f1-4325-9926-35966af7e14c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:17.383647 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c3c31-39f1-4325-9926-35966af7e14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:17.404008 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-f45c3c31-39f1-4325-9926-35966af7e14c None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:17.405393 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-f45c3c31-39f1-4325-9926-35966af7e14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:17.456288 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:17.472226 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1362/2725] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:02:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:17.482375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45272e01-eebe-4e03-92b4-1529546b972a admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:17.485177 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-45272e01-eebe-4e03-92b4-1529546b972a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:17.518178 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:17.549284 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45272e01-eebe-4e03-92b4-1529546b972a admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:17.553514 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd3cae-bd1f-4ad0-a5be-608213673274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:17.554224 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd3cae-bd1f-4ad0-a5be-608213673274 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:17.554529 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd3cae-bd1f-4ad0-a5be-608213673274 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:17.554830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd3cae-bd1f-4ad0-a5be-608213673274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:17.566591 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd3cae-bd1f-4ad0-a5be-608213673274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:17.568080 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd3cae-bd1f-4ad0-a5be-608213673274 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:17.578828 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1364/2726] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:02:18.009849 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c3c31-39f1-4325-9926-35966af7e14c None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:18.011319 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-f45c3c31-39f1-4325-9926-35966af7e14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:18.025592 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c3c31-39f1-4325-9926-35966af7e14c None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:18.030322 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e8cc3-130b-419e-b7e5-bb1b6ece2b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:18.031557 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e8cc3-130b-419e-b7e5-bb1b6ece2b29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:18.031889 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e8cc3-130b-419e-b7e5-bb1b6ece2b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:18.032224 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e8cc3-130b-419e-b7e5-bb1b6ece2b29 admin admin] PATH_INFO: `/v3/projects/6018b22d0d4c4999931ce8f1fc224c27` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:18.033390 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e8cc3-130b-419e-b7e5-bb1b6ece2b29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6018b22d0d4c4999931ce8f1fc224c27)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:18.035402 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e8cc3-130b-419e-b7e5-bb1b6ece2b29 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:18.192688 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-289e8cc3-130b-419e-b7e5-bb1b6ece2b29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6018b22d0d4c4999931ce8f1fc224c27', 'request_id': 'req-289e8cc3-130b-419e-b7e5-bb1b6ece2b29'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:18.199270 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-289e8cc3-130b-419e-b7e5-bb1b6ece2b29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6018b22d0d4c4999931ce8f1fc224c27', 'request_id': 'req-289e8cc3-130b-419e-b7e5-bb1b6ece2b29'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:18.211024 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-289e8cc3-130b-419e-b7e5-bb1b6ece2b29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6018b22d0d4c4999931ce8f1fc224c27', 'request_id': 'req-289e8cc3-130b-419e-b7e5-bb1b6ece2b29'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:18.361919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-289e8cc3-130b-419e-b7e5-bb1b6ece2b29 admin admin] The token cache is being invalidate because project 6018b22d0d4c4999931ce8f1fc224c27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:02:18.362208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-289e8cc3-130b-419e-b7e5-bb1b6ece2b29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:18.362851 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-289e8cc3-130b-419e-b7e5-bb1b6ece2b29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:18.364370 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1363/2727] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:02:18 2022] DELETE /identity/v3/projects/6018b22d0d4c4999931ce8f1fc224c27 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:21.879368 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8ee51-43ae-4e1f-b5c8-2ac7c72e4036 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:21.879928 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8ee51-43ae-4e1f-b5c8-2ac7c72e4036 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:21.879928 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8ee51-43ae-4e1f-b5c8-2ac7c72e4036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:21.931254 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-f4f8ee51-43ae-4e1f-b5c8-2ac7c72e4036 None None] MFA Rules not processed for user `9646662b0b3048f79ad5d2b9602f10a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:21.932486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f4f8ee51-43ae-4e1f-b5c8-2ac7c72e4036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:22.002485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1365/2728] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Aug 8 17:02:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:22.025663 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e8cc3-130b-419e-b7e5-bb1b6ece2b29 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:22.027213 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-289e8cc3-130b-419e-b7e5-bb1b6ece2b29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:22.131467 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e8cc3-130b-419e-b7e5-bb1b6ece2b29 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:22.136672 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa630413-316c-4ba2-811c-e88a2df82249 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c606decb832444e38a194b98f8d15f8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:22.137280 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa630413-316c-4ba2-811c-e88a2df82249 service swift] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:22.137556 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa630413-316c-4ba2-811c-e88a2df82249 service swift] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:22.137854 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa630413-316c-4ba2-811c-e88a2df82249 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:22.148037 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa630413-316c-4ba2-811c-e88a2df82249 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:22.149697 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa630413-316c-4ba2-811c-e88a2df82249 service swift] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:22.160868 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1364/2729] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Aug 8 17:02:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 17:02:23.193881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26234672-275c-4b2c-a9cd-10e1c2b15f8e None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:23.194410 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26234672-275c-4b2c-a9cd-10e1c2b15f8e None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:23.194410 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26234672-275c-4b2c-a9cd-10e1c2b15f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:23.214097 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-26234672-275c-4b2c-a9cd-10e1c2b15f8e None None] MFA Rules not processed for user `9646662b0b3048f79ad5d2b9602f10a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:23.215328 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-26234672-275c-4b2c-a9cd-10e1c2b15f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:23.223855 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1366/2730] 10.222.0.13 () {58 vars in 967 bytes} [Mon Aug 8 17:02:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:23.241855 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa630413-316c-4ba2-811c-e88a2df82249 service swift] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:23.253192 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa630413-316c-4ba2-811c-e88a2df82249 service swift] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:23.258664 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eff6f3-7286-4905-9441-6f3570c03bf8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c606decb832444e38a194b98f8d15f8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:23.259318 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eff6f3-7286-4905-9441-6f3570c03bf8 service swift] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:23.259569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eff6f3-7286-4905-9441-6f3570c03bf8 service swift] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:23.259861 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eff6f3-7286-4905-9441-6f3570c03bf8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:23.269146 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eff6f3-7286-4905-9441-6f3570c03bf8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:23.270782 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eff6f3-7286-4905-9441-6f3570c03bf8 service swift] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:23.279483 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1365/2731] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Aug 8 17:02:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 17:02:23.903918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3fef0-0526-427a-b942-4a5d305fcac1 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:23.904226 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3fef0-0526-427a-b942-4a5d305fcac1 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:23.904499 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3fef0-0526-427a-b942-4a5d305fcac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:23.926225 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-3cb3fef0-0526-427a-b942-4a5d305fcac1 None None] MFA Rules not processed for user `9646662b0b3048f79ad5d2b9602f10a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:23.927690 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-3cb3fef0-0526-427a-b942-4a5d305fcac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:23.935512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1367/2732] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Aug 8 17:02:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:23.947460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eff6f3-7286-4905-9441-6f3570c03bf8 service swift] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:23.958870 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eff6f3-7286-4905-9441-6f3570c03bf8 service swift] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:23.963880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b746d-db16-4da8-9a63-6a1f67d3daf2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c606decb832444e38a194b98f8d15f8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:23.964458 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b746d-db16-4da8-9a63-6a1f67d3daf2 service swift] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:23.964729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b746d-db16-4da8-9a63-6a1f67d3daf2 service swift] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:23.965106 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b746d-db16-4da8-9a63-6a1f67d3daf2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:23.974017 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b746d-db16-4da8-9a63-6a1f67d3daf2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:23.975702 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b746d-db16-4da8-9a63-6a1f67d3daf2 service swift] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:23.983783 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1366/2733] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Aug 8 17:02:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 17:02:24.513722 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26f30e-66fe-4b35-8087-22b9e32eb91c None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:24.514247 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26f30e-66fe-4b35-8087-22b9e32eb91c None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:24.514342 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26f30e-66fe-4b35-8087-22b9e32eb91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:24.541875 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-4c26f30e-66fe-4b35-8087-22b9e32eb91c None None] MFA Rules not processed for user `9646662b0b3048f79ad5d2b9602f10a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:24.543184 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-4c26f30e-66fe-4b35-8087-22b9e32eb91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:24.551152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1368/2734] 10.222.0.13 () {58 vars in 967 bytes} [Mon Aug 8 17:02:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:24.567636 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b746d-db16-4da8-9a63-6a1f67d3daf2 service swift] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:24.578924 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b746d-db16-4da8-9a63-6a1f67d3daf2 service swift] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:24.585403 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab6f50-949c-490e-9e7d-e53767433107 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c606decb832444e38a194b98f8d15f8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:24.586006 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab6f50-949c-490e-9e7d-e53767433107 service swift] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:24.586277 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab6f50-949c-490e-9e7d-e53767433107 service swift] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:24.586541 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab6f50-949c-490e-9e7d-e53767433107 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:24.595833 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab6f50-949c-490e-9e7d-e53767433107 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:24.598614 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab6f50-949c-490e-9e7d-e53767433107 service swift] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:24.614466 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1367/2735] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Aug 8 17:02:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 17:02:27.418063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26f30e-66fe-4b35-8087-22b9e32eb91c None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:27.419508 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-4c26f30e-66fe-4b35-8087-22b9e32eb91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:27.488742 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26f30e-66fe-4b35-8087-22b9e32eb91c None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:27.493400 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a444674-826a-4ca9-bdf5-3623e6e29480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:27.494043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a444674-826a-4ca9-bdf5-3623e6e29480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:27.494321 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a444674-826a-4ca9-bdf5-3623e6e29480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:27.494613 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a444674-826a-4ca9-bdf5-3623e6e29480 admin admin] PATH_INFO: `/v3/users/eda9c43329f54e19bead646695ba29d2` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:27.495922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a444674-826a-4ca9-bdf5-3623e6e29480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eda9c43329f54e19bead646695ba29d2)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:27.497879 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a444674-826a-4ca9-bdf5-3623e6e29480 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:27.522357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:27.702610 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-4a444674-826a-4ca9-bdf5-3623e6e29480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eda9c43329f54e19bead646695ba29d2'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:27.812085 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-4a444674-826a-4ca9-bdf5-3623e6e29480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eda9c43329f54e19bead646695ba29d2'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:27.812778 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-4a444674-826a-4ca9-bdf5-3623e6e29480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eda9c43329f54e19bead646695ba29d2'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:27.821568 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-4a444674-826a-4ca9-bdf5-3623e6e29480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eda9c43329f54e19bead646695ba29d2'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:27.836817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-4a444674-826a-4ca9-bdf5-3623e6e29480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eda9c43329f54e19bead646695ba29d2'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:27.846602 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1369/2736] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:02:27 2022] DELETE /identity/v3/users/eda9c43329f54e19bead646695ba29d2 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:02:27.863409 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa1c78-2efe-4c0f-b001-30cbd75294d2 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:27.864249 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa1c78-2efe-4c0f-b001-30cbd75294d2 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:27.864249 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa1c78-2efe-4c0f-b001-30cbd75294d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:27.872962 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:27.885322 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-04aa1c78-2efe-4c0f-b001-30cbd75294d2 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:27.886697 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-04aa1c78-2efe-4c0f-b001-30cbd75294d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:27.982991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1368/2737] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:02:27.999208 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a444674-826a-4ca9-bdf5-3623e6e29480 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:28.001673 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-4a444674-826a-4ca9-bdf5-3623e6e29480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:28.081068 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a444674-826a-4ca9-bdf5-3623e6e29480 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:28.087169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d14b5-8f34-4423-94ca-c347d32b9760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:28.088007 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d14b5-8f34-4423-94ca-c347d32b9760 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:28.088412 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d14b5-8f34-4423-94ca-c347d32b9760 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:28.088886 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d14b5-8f34-4423-94ca-c347d32b9760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:28.101832 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d14b5-8f34-4423-94ca-c347d32b9760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:28.103372 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d14b5-8f34-4423-94ca-c347d32b9760 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:28.114984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1370/2738] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:02:28.442836 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa1c78-2efe-4c0f-b001-30cbd75294d2 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:28.442836 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-04aa1c78-2efe-4c0f-b001-30cbd75294d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:28.453473 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa1c78-2efe-4c0f-b001-30cbd75294d2 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:28.459448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8123a78-ef29-4f9e-b0e0-d2bc415c153d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:28.460342 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8123a78-ef29-4f9e-b0e0-d2bc415c153d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:28.460508 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8123a78-ef29-4f9e-b0e0-d2bc415c153d admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:28.460740 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8123a78-ef29-4f9e-b0e0-d2bc415c153d admin admin] PATH_INFO: `/v3/projects/4fb2e613a7774f0291ba9cccc18cfb31` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:28.462374 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8123a78-ef29-4f9e-b0e0-d2bc415c153d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fb2e613a7774f0291ba9cccc18cfb31)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:28.464461 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8123a78-ef29-4f9e-b0e0-d2bc415c153d admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:28.634485 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-a8123a78-ef29-4f9e-b0e0-d2bc415c153d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fb2e613a7774f0291ba9cccc18cfb31', 'request_id': 'req-a8123a78-ef29-4f9e-b0e0-d2bc415c153d'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:28.641386 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-a8123a78-ef29-4f9e-b0e0-d2bc415c153d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fb2e613a7774f0291ba9cccc18cfb31', 'request_id': 'req-a8123a78-ef29-4f9e-b0e0-d2bc415c153d'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:28.648884 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-a8123a78-ef29-4f9e-b0e0-d2bc415c153d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fb2e613a7774f0291ba9cccc18cfb31', 'request_id': 'req-a8123a78-ef29-4f9e-b0e0-d2bc415c153d'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:28.765533 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-a8123a78-ef29-4f9e-b0e0-d2bc415c153d admin admin] The token cache is being invalidate because project 4fb2e613a7774f0291ba9cccc18cfb31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:02:28.765765 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-a8123a78-ef29-4f9e-b0e0-d2bc415c153d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:28.766359 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-a8123a78-ef29-4f9e-b0e0-d2bc415c153d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:28.768336 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1369/2739] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:02:28 2022] DELETE /identity/v3/projects/4fb2e613a7774f0291ba9cccc18cfb31 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:02:35.759454 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d14b5-8f34-4423-94ca-c347d32b9760 service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:35.760957 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-250d14b5-8f34-4423-94ca-c347d32b9760 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:35.772587 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d14b5-8f34-4423-94ca-c347d32b9760 service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:35.821454 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a36d8-0232-4166-adc4-172384a9d7dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:35.822058 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a36d8-0232-4166-adc4-172384a9d7dc service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:35.822335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a36d8-0232-4166-adc4-172384a9d7dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:35.822629 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a36d8-0232-4166-adc4-172384a9d7dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:35.824368 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-7d3a36d8-0232-4166-adc4-172384a9d7dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:35.883341 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:35.887237 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a36d8-0232-4166-adc4-172384a9d7dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:35.888663 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a36d8-0232-4166-adc4-172384a9d7dc service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:35.899145 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1371/2740] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:02:38.762142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=52,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:39.648527 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8123a78-ef29-4f9e-b0e0-d2bc415c153d admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:39.650079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-a8123a78-ef29-4f9e-b0e0-d2bc415c153d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:39.664797 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8123a78-ef29-4f9e-b0e0-d2bc415c153d admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:39.669419 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54f00c-4c3e-42aa-b22d-34d603d15c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:39.670261 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54f00c-4c3e-42aa-b22d-34d603d15c62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:39.670539 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54f00c-4c3e-42aa-b22d-34d603d15c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:39.670841 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54f00c-4c3e-42aa-b22d-34d603d15c62 admin admin] PATH_INFO: `/v3/users/d6d78cbc233a498db3a9268a4b93d737` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:39.671977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54f00c-4c3e-42aa-b22d-34d603d15c62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6d78cbc233a498db3a9268a4b93d737)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:39.673887 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54f00c-4c3e-42aa-b22d-34d603d15c62 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:39.864876 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-4f54f00c-4c3e-42aa-b22d-34d603d15c62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6d78cbc233a498db3a9268a4b93d737'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:39.976927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-4f54f00c-4c3e-42aa-b22d-34d603d15c62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6d78cbc233a498db3a9268a4b93d737'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:39.977548 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-4f54f00c-4c3e-42aa-b22d-34d603d15c62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6d78cbc233a498db3a9268a4b93d737'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:39.985337 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-4f54f00c-4c3e-42aa-b22d-34d603d15c62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6d78cbc233a498db3a9268a4b93d737'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:39.990723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-4f54f00c-4c3e-42aa-b22d-34d603d15c62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6d78cbc233a498db3a9268a4b93d737'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:40.005179 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1370/2741] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:02:39 2022] DELETE /identity/v3/users/d6d78cbc233a498db3a9268a4b93d737 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:40.265453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a36d8-0232-4166-adc4-172384a9d7dc service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:40.276115 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a36d8-0232-4166-adc4-172384a9d7dc service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:40.337380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f0b8b-50ff-4608-87f1-a70568b6fc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:40.338015 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f0b8b-50ff-4608-87f1-a70568b6fc6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:40.338294 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f0b8b-50ff-4608-87f1-a70568b6fc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:40.339897 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f0b8b-50ff-4608-87f1-a70568b6fc6f admin admin] PATH_INFO: `/v3/projects/fa4be8df3fe34d71a3c3d91995c1edae` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:40.339897 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f0b8b-50ff-4608-87f1-a70568b6fc6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa4be8df3fe34d71a3c3d91995c1edae)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:40.341768 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f0b8b-50ff-4608-87f1-a70568b6fc6f admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:40.448296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-797f0b8b-50ff-4608-87f1-a70568b6fc6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa4be8df3fe34d71a3c3d91995c1edae', 'request_id': 'req-797f0b8b-50ff-4608-87f1-a70568b6fc6f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:40.493485 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-797f0b8b-50ff-4608-87f1-a70568b6fc6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa4be8df3fe34d71a3c3d91995c1edae', 'request_id': 'req-797f0b8b-50ff-4608-87f1-a70568b6fc6f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:40.500119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-797f0b8b-50ff-4608-87f1-a70568b6fc6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa4be8df3fe34d71a3c3d91995c1edae', 'request_id': 'req-797f0b8b-50ff-4608-87f1-a70568b6fc6f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:40.510584 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-797f0b8b-50ff-4608-87f1-a70568b6fc6f admin admin] The token cache is being invalidate because project fa4be8df3fe34d71a3c3d91995c1edae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:02:40.510875 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-797f0b8b-50ff-4608-87f1-a70568b6fc6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:40.511495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-797f0b8b-50ff-4608-87f1-a70568b6fc6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:40.512982 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1372/2742] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:02:40 2022] DELETE /identity/v3/projects/fa4be8df3fe34d71a3c3d91995c1edae => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:40.517949 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ccf24-45a8-4be5-89e6-2750abbefe93 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:40.518891 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ccf24-45a8-4be5-89e6-2750abbefe93 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:40.519151 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ccf24-45a8-4be5-89e6-2750abbefe93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:40.539314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-d02ccf24-45a8-4be5-89e6-2750abbefe93 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:40.540516 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d02ccf24-45a8-4be5-89e6-2750abbefe93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:40.601368 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1371/2743] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:40.609186 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd653a-abd1-427b-becd-273298b360a9 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:40.609467 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd653a-abd1-427b-becd-273298b360a9 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:40.609743 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd653a-abd1-427b-becd-273298b360a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:40.629584 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-8fbd653a-abd1-427b-becd-273298b360a9 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:40.630809 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-8fbd653a-abd1-427b-becd-273298b360a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:40.638871 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1373/2744] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:40.643579 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ccf24-45a8-4be5-89e6-2750abbefe93 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:40.653447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ccf24-45a8-4be5-89e6-2750abbefe93 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:40.658015 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d4c91-931d-44d6-92bb-0cc6cfd1bbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:40.658624 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d4c91-931d-44d6-92bb-0cc6cfd1bbf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:40.659514 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d4c91-931d-44d6-92bb-0cc6cfd1bbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:40.659779 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d4c91-931d-44d6-92bb-0cc6cfd1bbf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:40.660447 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d4c91-931d-44d6-92bb-0cc6cfd1bbf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:02:40.660772 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d4c91-931d-44d6-92bb-0cc6cfd1bbf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:40.662354 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d4c91-931d-44d6-92bb-0cc6cfd1bbf5 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:40.673804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1372/2745] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 17:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:02:40.678734 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd653a-abd1-427b-becd-273298b360a9 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:40.688181 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd653a-abd1-427b-becd-273298b360a9 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:40.693351 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9f5f5-e3b2-454b-a8e9-8c5d56860870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:40.693938 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9f5f5-e3b2-454b-a8e9-8c5d56860870 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:40.694212 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9f5f5-e3b2-454b-a8e9-8c5d56860870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:40.694478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9f5f5-e3b2-454b-a8e9-8c5d56860870 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:40.695121 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9f5f5-e3b2-454b-a8e9-8c5d56860870 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:40.696991 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9f5f5-e3b2-454b-a8e9-8c5d56860870 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:40.780592 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1374/2746] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:02:40 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:02:40.785676 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d4c91-931d-44d6-92bb-0cc6cfd1bbf5 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:40.795729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d4c91-931d-44d6-92bb-0cc6cfd1bbf5 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:40.852499 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eac097-6ab6-4380-86fc-50429857daf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:40.853091 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eac097-6ab6-4380-86fc-50429857daf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:40.853365 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eac097-6ab6-4380-86fc-50429857daf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:40.853630 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eac097-6ab6-4380-86fc-50429857daf8 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:40.854246 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eac097-6ab6-4380-86fc-50429857daf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:40.856184 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eac097-6ab6-4380-86fc-50429857daf8 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:40.864073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:40.946290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1373/2747] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:02:40 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:02:40.950338 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9f5f5-e3b2-454b-a8e9-8c5d56860870 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:40.962321 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9f5f5-e3b2-454b-a8e9-8c5d56860870 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:40.967174 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f698c3ca-f451-4411-adb7-7d2c3a21dd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:40.967866 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698c3ca-f451-4411-adb7-7d2c3a21dd55 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:40.968055 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698c3ca-f451-4411-adb7-7d2c3a21dd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:40.968327 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698c3ca-f451-4411-adb7-7d2c3a21dd55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:40.968985 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f698c3ca-f451-4411-adb7-7d2c3a21dd55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:40.970574 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f698c3ca-f451-4411-adb7-7d2c3a21dd55 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:40.981176 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1375/2748] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:02:40.985738 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eac097-6ab6-4380-86fc-50429857daf8 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:40.996544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eac097-6ab6-4380-86fc-50429857daf8 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:41.001698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf95f91-7470-4bd4-a502-3c0fcda9efed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:41.003269 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf95f91-7470-4bd4-a502-3c0fcda9efed admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:41.003767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf95f91-7470-4bd4-a502-3c0fcda9efed admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:41.004037 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf95f91-7470-4bd4-a502-3c0fcda9efed admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:41.004730 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf95f91-7470-4bd4-a502-3c0fcda9efed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:41.006352 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf95f91-7470-4bd4-a502-3c0fcda9efed admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:41.016883 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1374/2749] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:02:41.021833 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f698c3ca-f451-4411-adb7-7d2c3a21dd55 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:41.031927 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f698c3ca-f451-4411-adb7-7d2c3a21dd55 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:41.036563 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c14e31-49a3-44e2-a2ba-960ba637e47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:41.037213 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c14e31-49a3-44e2-a2ba-960ba637e47c admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:41.037491 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c14e31-49a3-44e2-a2ba-960ba637e47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:41.037762 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c14e31-49a3-44e2-a2ba-960ba637e47c admin admin] PATH_INFO: `/v3/projects/765343db9d96417b83e54011d40ffd76/users/27ac81c1f5b7414fb97aa212845d9519/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:41.052187 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c14e31-49a3-44e2-a2ba-960ba637e47c admin admin] RBAC: Authorizing `identity:create_grant(project_id=765343db9d96417b83e54011d40ffd76, user_id=27ac81c1f5b7414fb97aa212845d9519, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:41.054855 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c14e31-49a3-44e2-a2ba-960ba637e47c admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:41.123555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1376/2750] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:02:41 2022] PUT /identity/v3/projects/765343db9d96417b83e54011d40ffd76/users/27ac81c1f5b7414fb97aa212845d9519/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:41.135962 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3fd58-1054-4411-9a5d-49de30d301d2 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:41.136243 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3fd58-1054-4411-9a5d-49de30d301d2 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:41.136515 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3fd58-1054-4411-9a5d-49de30d301d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:41.167827 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-5da3fd58-1054-4411-9a5d-49de30d301d2 None None] MFA Rules not processed for user `27ac81c1f5b7414fb97aa212845d9519`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:41.169049 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-5da3fd58-1054-4411-9a5d-49de30d301d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:41.233366 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1375/2751] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:02:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:41.238638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c14e31-49a3-44e2-a2ba-960ba637e47c admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:41.260287 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:41.305734 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c14e31-49a3-44e2-a2ba-960ba637e47c admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:41.310805 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab66a28-fdf2-429b-aefe-6e16904ffd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:41.313314 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab66a28-fdf2-429b-aefe-6e16904ffd16 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:41.313314 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab66a28-fdf2-429b-aefe-6e16904ffd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:41.313314 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab66a28-fdf2-429b-aefe-6e16904ffd16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:41.313314 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab66a28-fdf2-429b-aefe-6e16904ffd16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:41.314490 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab66a28-fdf2-429b-aefe-6e16904ffd16 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:41.407097 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1377/2752] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:02:41 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:02:41.411641 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3fd58-1054-4411-9a5d-49de30d301d2 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:41.422465 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3fd58-1054-4411-9a5d-49de30d301d2 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:41.477679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17373ce-e79e-4897-bc8c-f7dcc49bf4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:41.478315 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17373ce-e79e-4897-bc8c-f7dcc49bf4df admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:41.478544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17373ce-e79e-4897-bc8c-f7dcc49bf4df admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:41.478835 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17373ce-e79e-4897-bc8c-f7dcc49bf4df admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:41.479461 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17373ce-e79e-4897-bc8c-f7dcc49bf4df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:41.481362 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17373ce-e79e-4897-bc8c-f7dcc49bf4df admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:41.588909 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1376/2753] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:02:41 2022] POST /identity/v3/users => generated 299 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:02:41.592956 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab66a28-fdf2-429b-aefe-6e16904ffd16 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:41.604702 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab66a28-fdf2-429b-aefe-6e16904ffd16 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:41.609335 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093ba1d-eff6-4f49-920c-6ecbae62c1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:41.609925 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093ba1d-eff6-4f49-920c-6ecbae62c1ce admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:41.610197 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093ba1d-eff6-4f49-920c-6ecbae62c1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:41.610606 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093ba1d-eff6-4f49-920c-6ecbae62c1ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:41.611245 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0093ba1d-eff6-4f49-920c-6ecbae62c1ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:41.614014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0093ba1d-eff6-4f49-920c-6ecbae62c1ce admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:41.624932 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1378/2754] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:02:41.629986 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17373ce-e79e-4897-bc8c-f7dcc49bf4df admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:41.640400 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17373ce-e79e-4897-bc8c-f7dcc49bf4df admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:41.645366 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004bad6-2fe6-48c4-9918-489ed43aa523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:41.646019 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004bad6-2fe6-48c4-9918-489ed43aa523 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:41.646295 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004bad6-2fe6-48c4-9918-489ed43aa523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:41.646674 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004bad6-2fe6-48c4-9918-489ed43aa523 admin admin] PATH_INFO: `/v3/projects/43dccfd8653e4e919418231b04973627/users/ec32d99bbc4c43e4b71174fbfd2e1067/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:41.664167 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8004bad6-2fe6-48c4-9918-489ed43aa523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43dccfd8653e4e919418231b04973627, user_id=ec32d99bbc4c43e4b71174fbfd2e1067, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:41.666886 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8004bad6-2fe6-48c4-9918-489ed43aa523 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:41.771846 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1377/2755] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:02:41 2022] PUT /identity/v3/projects/43dccfd8653e4e919418231b04973627/users/ec32d99bbc4c43e4b71174fbfd2e1067/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:41.775875 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093ba1d-eff6-4f49-920c-6ecbae62c1ce admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:41.786633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093ba1d-eff6-4f49-920c-6ecbae62c1ce admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:41.841505 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615fdca-2a43-4628-9605-6d3664bb16a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:41.842022 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615fdca-2a43-4628-9605-6d3664bb16a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:41.842300 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615fdca-2a43-4628-9605-6d3664bb16a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:41.842575 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615fdca-2a43-4628-9605-6d3664bb16a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:41.843217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615fdca-2a43-4628-9605-6d3664bb16a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:41.844780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615fdca-2a43-4628-9605-6d3664bb16a0 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:41.855042 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1379/2756] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:02:41.859316 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004bad6-2fe6-48c4-9918-489ed43aa523 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:41.870266 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004bad6-2fe6-48c4-9918-489ed43aa523 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:41.874874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be13b78-337a-4635-aac7-a3671b6cbf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:41.875536 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be13b78-337a-4635-aac7-a3671b6cbf3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:41.875796 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be13b78-337a-4635-aac7-a3671b6cbf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:41.876064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be13b78-337a-4635-aac7-a3671b6cbf3a admin admin] PATH_INFO: `/v3/projects/43dccfd8653e4e919418231b04973627/users/ec32d99bbc4c43e4b71174fbfd2e1067/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:41.878056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be13b78-337a-4635-aac7-a3671b6cbf3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=43dccfd8653e4e919418231b04973627, user_id=ec32d99bbc4c43e4b71174fbfd2e1067, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:41.880712 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be13b78-337a-4635-aac7-a3671b6cbf3a admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:41.890859 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1378/2757] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:02:41 2022] PUT /identity/v3/projects/43dccfd8653e4e919418231b04973627/users/ec32d99bbc4c43e4b71174fbfd2e1067/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:41.901353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c4996-1b78-4c88-a8ea-140a30f53c81 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:41.901641 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c4996-1b78-4c88-a8ea-140a30f53c81 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:41.901920 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c4996-1b78-4c88-a8ea-140a30f53c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:41.931205 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-061c4996-1b78-4c88-a8ea-140a30f53c81 None None] MFA Rules not processed for user `ec32d99bbc4c43e4b71174fbfd2e1067`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:41.932534 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-061c4996-1b78-4c88-a8ea-140a30f53c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:42.005554 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1380/2758] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:02:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:42.017027 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be13b78-337a-4635-aac7-a3671b6cbf3a admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:42.019347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-3be13b78-337a-4635-aac7-a3671b6cbf3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:42.042275 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:42.079692 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be13b78-337a-4635-aac7-a3671b6cbf3a admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:42.084417 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1215b4-d5d1-4898-93d0-88b79e0cc1b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:42.085018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1215b4-d5d1-4898-93d0-88b79e0cc1b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:42.085290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1215b4-d5d1-4898-93d0-88b79e0cc1b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:42.085560 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1215b4-d5d1-4898-93d0-88b79e0cc1b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:42.094685 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1215b4-d5d1-4898-93d0-88b79e0cc1b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:42.096059 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1215b4-d5d1-4898-93d0-88b79e0cc1b9 service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:42.107463 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1379/2759] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:02:42.135445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c4996-1b78-4c88-a8ea-140a30f53c81 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:42.153512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:42.198763 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c4996-1b78-4c88-a8ea-140a30f53c81 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:42.203473 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d9549-0455-4ad3-9ef7-442fc0d35059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:42.204100 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d9549-0455-4ad3-9ef7-442fc0d35059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:42.204377 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d9549-0455-4ad3-9ef7-442fc0d35059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:42.204644 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d9549-0455-4ad3-9ef7-442fc0d35059 admin admin] PATH_INFO: `/v3/users/27ac81c1f5b7414fb97aa212845d9519` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:42.206387 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d9549-0455-4ad3-9ef7-442fc0d35059 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27ac81c1f5b7414fb97aa212845d9519)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:42.208343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d9549-0455-4ad3-9ef7-442fc0d35059 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:42.381927 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-ea0d9549-0455-4ad3-9ef7-442fc0d35059 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27ac81c1f5b7414fb97aa212845d9519'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:42.533283 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-ea0d9549-0455-4ad3-9ef7-442fc0d35059 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27ac81c1f5b7414fb97aa212845d9519'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:42.534004 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-ea0d9549-0455-4ad3-9ef7-442fc0d35059 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27ac81c1f5b7414fb97aa212845d9519'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:42.541512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-ea0d9549-0455-4ad3-9ef7-442fc0d35059 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27ac81c1f5b7414fb97aa212845d9519'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:42.553436 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-ea0d9549-0455-4ad3-9ef7-442fc0d35059 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27ac81c1f5b7414fb97aa212845d9519'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:42.559333 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1381/2760] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:02:42 2022] DELETE /identity/v3/users/27ac81c1f5b7414fb97aa212845d9519 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:42.564253 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c068ad-d81a-41bb-a458-4b28e1321965 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:42.564542 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c068ad-d81a-41bb-a458-4b28e1321965 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:42.565334 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c068ad-d81a-41bb-a458-4b28e1321965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:42.585018 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-e0c068ad-d81a-41bb-a458-4b28e1321965 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:42.586346 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-e0c068ad-d81a-41bb-a458-4b28e1321965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:42.647678 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1380/2761] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:42.657432 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d9549-0455-4ad3-9ef7-442fc0d35059 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:42.659307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ea0d9549-0455-4ad3-9ef7-442fc0d35059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:42.721302 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d9549-0455-4ad3-9ef7-442fc0d35059 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:42.725835 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7afe2-43d0-4dbe-90d7-faca51d750a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:42.726449 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7afe2-43d0-4dbe-90d7-faca51d750a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:42.726758 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7afe2-43d0-4dbe-90d7-faca51d750a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:42.727026 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7afe2-43d0-4dbe-90d7-faca51d750a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:42.736528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f7afe2-43d0-4dbe-90d7-faca51d750a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:42.738997 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f7afe2-43d0-4dbe-90d7-faca51d750a1 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:42.748640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1382/2762] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:02:43.181897 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c068ad-d81a-41bb-a458-4b28e1321965 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:43.183347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-e0c068ad-d81a-41bb-a458-4b28e1321965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:43.196114 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c068ad-d81a-41bb-a458-4b28e1321965 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:43.201228 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa727c-7818-440d-aed7-71256a6e9ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:43.201874 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa727c-7818-440d-aed7-71256a6e9ab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:43.202149 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa727c-7818-440d-aed7-71256a6e9ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:43.202417 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa727c-7818-440d-aed7-71256a6e9ab7 admin admin] PATH_INFO: `/v3/projects/765343db9d96417b83e54011d40ffd76` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:43.203588 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa727c-7818-440d-aed7-71256a6e9ab7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=765343db9d96417b83e54011d40ffd76)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:43.205540 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa727c-7818-440d-aed7-71256a6e9ab7 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:43.322759 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f2aa727c-7818-440d-aed7-71256a6e9ab7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '765343db9d96417b83e54011d40ffd76', 'request_id': 'req-f2aa727c-7818-440d-aed7-71256a6e9ab7'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:43.328282 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f2aa727c-7818-440d-aed7-71256a6e9ab7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '765343db9d96417b83e54011d40ffd76', 'request_id': 'req-f2aa727c-7818-440d-aed7-71256a6e9ab7'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:43.337944 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f2aa727c-7818-440d-aed7-71256a6e9ab7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '765343db9d96417b83e54011d40ffd76', 'request_id': 'req-f2aa727c-7818-440d-aed7-71256a6e9ab7'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:43.424352 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f2aa727c-7818-440d-aed7-71256a6e9ab7 admin admin] The token cache is being invalidate because project 765343db9d96417b83e54011d40ffd76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:02:43.424578 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f2aa727c-7818-440d-aed7-71256a6e9ab7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:43.425194 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-f2aa727c-7818-440d-aed7-71256a6e9ab7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:43.426724 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1381/2763] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:02:43 2022] DELETE /identity/v3/projects/765343db9d96417b83e54011d40ffd76 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:43.430853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7afe2-43d0-4dbe-90d7-faca51d750a1 service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:43.440996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7afe2-43d0-4dbe-90d7-faca51d750a1 service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:43.475830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:43.498637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1e1ea-96fe-4e31-b735-aa687c91bbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:43.499277 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1e1ea-96fe-4e31-b735-aa687c91bbf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:43.499573 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1e1ea-96fe-4e31-b735-aa687c91bbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:43.499843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1e1ea-96fe-4e31-b735-aa687c91bbf9 admin admin] PATH_INFO: `/v3/users/ec32d99bbc4c43e4b71174fbfd2e1067` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:43.500968 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff1e1ea-96fe-4e31-b735-aa687c91bbf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec32d99bbc4c43e4b71174fbfd2e1067)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:43.502911 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff1e1ea-96fe-4e31-b735-aa687c91bbf9 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:43.710581 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-dff1e1ea-96fe-4e31-b735-aa687c91bbf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec32d99bbc4c43e4b71174fbfd2e1067'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:43.782859 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-dff1e1ea-96fe-4e31-b735-aa687c91bbf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec32d99bbc4c43e4b71174fbfd2e1067'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:43.783503 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-dff1e1ea-96fe-4e31-b735-aa687c91bbf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec32d99bbc4c43e4b71174fbfd2e1067'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:43.792185 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-dff1e1ea-96fe-4e31-b735-aa687c91bbf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec32d99bbc4c43e4b71174fbfd2e1067'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:43.804550 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-dff1e1ea-96fe-4e31-b735-aa687c91bbf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec32d99bbc4c43e4b71174fbfd2e1067'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:43.810974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1383/2764] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:02:43 2022] DELETE /identity/v3/users/ec32d99bbc4c43e4b71174fbfd2e1067 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:44.232424 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa727c-7818-440d-aed7-71256a6e9ab7 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:44.234054 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-f2aa727c-7818-440d-aed7-71256a6e9ab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:44.248071 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa727c-7818-440d-aed7-71256a6e9ab7 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:44.295651 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:44.309122 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6eb9f-ee3c-4a8d-9cd5-63b7cb04ca0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:44.310044 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6eb9f-ee3c-4a8d-9cd5-63b7cb04ca0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:44.310320 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6eb9f-ee3c-4a8d-9cd5-63b7cb04ca0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:44.310624 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6eb9f-ee3c-4a8d-9cd5-63b7cb04ca0c admin admin] PATH_INFO: `/v3/projects/43dccfd8653e4e919418231b04973627` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:44.311752 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6eb9f-ee3c-4a8d-9cd5-63b7cb04ca0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=43dccfd8653e4e919418231b04973627)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:44.313717 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6eb9f-ee3c-4a8d-9cd5-63b7cb04ca0c admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:44.436851 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c6d6eb9f-ee3c-4a8d-9cd5-63b7cb04ca0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43dccfd8653e4e919418231b04973627', 'request_id': 'req-c6d6eb9f-ee3c-4a8d-9cd5-63b7cb04ca0c'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:44.442474 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c6d6eb9f-ee3c-4a8d-9cd5-63b7cb04ca0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43dccfd8653e4e919418231b04973627', 'request_id': 'req-c6d6eb9f-ee3c-4a8d-9cd5-63b7cb04ca0c'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:44.451030 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c6d6eb9f-ee3c-4a8d-9cd5-63b7cb04ca0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43dccfd8653e4e919418231b04973627', 'request_id': 'req-c6d6eb9f-ee3c-4a8d-9cd5-63b7cb04ca0c'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:44.539394 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c6d6eb9f-ee3c-4a8d-9cd5-63b7cb04ca0c admin admin] The token cache is being invalidate because project 43dccfd8653e4e919418231b04973627 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:02:44.539600 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c6d6eb9f-ee3c-4a8d-9cd5-63b7cb04ca0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:44.540200 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-c6d6eb9f-ee3c-4a8d-9cd5-63b7cb04ca0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:44.541849 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1382/2765] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:02:44 2022] DELETE /identity/v3/projects/43dccfd8653e4e919418231b04973627 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:44.546828 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4ff03-c696-40f7-9a0e-7d1a18168680 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:44.547307 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4ff03-c696-40f7-9a0e-7d1a18168680 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:44.547412 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4ff03-c696-40f7-9a0e-7d1a18168680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:44.570095 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-e7b4ff03-c696-40f7-9a0e-7d1a18168680 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:44.571325 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-e7b4ff03-c696-40f7-9a0e-7d1a18168680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:44.630593 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1384/2766] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:02:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:44.639929 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e7486-8015-41bb-8f24-f88eaf0bc092 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:44.640206 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e7486-8015-41bb-8f24-f88eaf0bc092 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:44.640806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e7486-8015-41bb-8f24-f88eaf0bc092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:44.660123 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-981e7486-8015-41bb-8f24-f88eaf0bc092 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:44.661821 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-981e7486-8015-41bb-8f24-f88eaf0bc092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:44.669912 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1383/2767] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:02:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:44.674182 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4ff03-c696-40f7-9a0e-7d1a18168680 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:44.684942 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4ff03-c696-40f7-9a0e-7d1a18168680 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:44.689501 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b554e29-4bf7-4a55-b222-0f4c86392fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:44.690103 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b554e29-4bf7-4a55-b222-0f4c86392fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:44.690375 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b554e29-4bf7-4a55-b222-0f4c86392fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:44.690672 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b554e29-4bf7-4a55-b222-0f4c86392fc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:44.691314 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b554e29-4bf7-4a55-b222-0f4c86392fc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:02:44.691608 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b554e29-4bf7-4a55-b222-0f4c86392fc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:44.693698 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b554e29-4bf7-4a55-b222-0f4c86392fc9 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:44.705615 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1385/2768] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 17:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:02:44.710160 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e7486-8015-41bb-8f24-f88eaf0bc092 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:44.719936 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e7486-8015-41bb-8f24-f88eaf0bc092 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:44.725973 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa6b39-b96f-44f3-b7db-9987258c992e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:44.726623 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa6b39-b96f-44f3-b7db-9987258c992e admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:44.726907 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa6b39-b96f-44f3-b7db-9987258c992e admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:44.727164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa6b39-b96f-44f3-b7db-9987258c992e admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:44.727789 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa6b39-b96f-44f3-b7db-9987258c992e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:44.729706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa6b39-b96f-44f3-b7db-9987258c992e admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:44.821029 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1384/2769] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:02:44 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:02:44.825104 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b554e29-4bf7-4a55-b222-0f4c86392fc9 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:44.835792 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b554e29-4bf7-4a55-b222-0f4c86392fc9 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:44.880115 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:44.894934 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f042c-5857-41ec-80b8-0fe2db3664a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:44.895536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f042c-5857-41ec-80b8-0fe2db3664a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:44.896327 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f042c-5857-41ec-80b8-0fe2db3664a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:44.897057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f042c-5857-41ec-80b8-0fe2db3664a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:44.897719 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f042c-5857-41ec-80b8-0fe2db3664a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:44.899649 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f042c-5857-41ec-80b8-0fe2db3664a8 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:45.019402 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1386/2770] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:02:44 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:02:45.023552 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa6b39-b96f-44f3-b7db-9987258c992e admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:45.033592 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa6b39-b96f-44f3-b7db-9987258c992e admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:45.038412 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c88037d-bf59-455b-9944-e8b0f4c9814e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:45.039030 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88037d-bf59-455b-9944-e8b0f4c9814e admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:45.039301 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88037d-bf59-455b-9944-e8b0f4c9814e admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:45.039562 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88037d-bf59-455b-9944-e8b0f4c9814e admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:45.040188 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c88037d-bf59-455b-9944-e8b0f4c9814e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:45.041760 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c88037d-bf59-455b-9944-e8b0f4c9814e admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:45.051566 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1385/2771] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:02:45.056525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f042c-5857-41ec-80b8-0fe2db3664a8 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:45.067285 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f042c-5857-41ec-80b8-0fe2db3664a8 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:45.072837 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912b5c5-f79f-4187-ba39-77d0df5b8eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:45.073517 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912b5c5-f79f-4187-ba39-77d0df5b8eae admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:45.073700 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912b5c5-f79f-4187-ba39-77d0df5b8eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:45.073991 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912b5c5-f79f-4187-ba39-77d0df5b8eae admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:45.074656 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912b5c5-f79f-4187-ba39-77d0df5b8eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:45.076224 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912b5c5-f79f-4187-ba39-77d0df5b8eae admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:45.087384 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1387/2772] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:02:45.091569 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c88037d-bf59-455b-9944-e8b0f4c9814e admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:45.101967 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c88037d-bf59-455b-9944-e8b0f4c9814e admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:45.106420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee656002-145a-4aee-bb57-7520a74053b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:45.107083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee656002-145a-4aee-bb57-7520a74053b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:45.107356 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee656002-145a-4aee-bb57-7520a74053b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:45.107619 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee656002-145a-4aee-bb57-7520a74053b3 admin admin] PATH_INFO: `/v3/projects/55d9a4c7ea23452eb70340774386e071/users/9aea4e5113184f798b8f1e9cc7bf4b55/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:45.122882 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee656002-145a-4aee-bb57-7520a74053b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55d9a4c7ea23452eb70340774386e071, user_id=9aea4e5113184f798b8f1e9cc7bf4b55, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:45.125519 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee656002-145a-4aee-bb57-7520a74053b3 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:45.234305 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1386/2773] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:02:45 2022] PUT /identity/v3/projects/55d9a4c7ea23452eb70340774386e071/users/9aea4e5113184f798b8f1e9cc7bf4b55/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:45.245514 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8c6ac-cc19-4d5f-8846-2d491d287725 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:45.245798 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8c6ac-cc19-4d5f-8846-2d491d287725 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:45.246125 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8c6ac-cc19-4d5f-8846-2d491d287725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:45.276008 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-39e8c6ac-cc19-4d5f-8846-2d491d287725 None None] MFA Rules not processed for user `9aea4e5113184f798b8f1e9cc7bf4b55`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:45.277361 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-39e8c6ac-cc19-4d5f-8846-2d491d287725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:45.346503 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1388/2774] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:02:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:45.352164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee656002-145a-4aee-bb57-7520a74053b3 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:45.414221 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee656002-145a-4aee-bb57-7520a74053b3 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:45.419224 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d321ac-b1f9-4e3e-abfc-27736a272aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:45.419848 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d321ac-b1f9-4e3e-abfc-27736a272aa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:45.420127 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d321ac-b1f9-4e3e-abfc-27736a272aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:45.420396 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d321ac-b1f9-4e3e-abfc-27736a272aa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:45.421065 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d321ac-b1f9-4e3e-abfc-27736a272aa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:45.423179 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d321ac-b1f9-4e3e-abfc-27736a272aa6 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:45.516414 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:45.524198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1387/2775] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:02:45 2022] POST /identity/v3/projects => generated 387 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:02:45.528405 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8c6ac-cc19-4d5f-8846-2d491d287725 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:45.540430 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8c6ac-cc19-4d5f-8846-2d491d287725 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:45.597151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f6d3e-ee6a-46e0-8873-b22f5d69acde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:45.597813 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f6d3e-ee6a-46e0-8873-b22f5d69acde admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:45.597944 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f6d3e-ee6a-46e0-8873-b22f5d69acde admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:45.598204 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f6d3e-ee6a-46e0-8873-b22f5d69acde admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:45.600020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f6d3e-ee6a-46e0-8873-b22f5d69acde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:45.601923 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f6d3e-ee6a-46e0-8873-b22f5d69acde admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:45.697267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1389/2776] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:02:45 2022] POST /identity/v3/users => generated 300 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:02:45.701314 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d321ac-b1f9-4e3e-abfc-27736a272aa6 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:45.712357 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d321ac-b1f9-4e3e-abfc-27736a272aa6 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:45.717074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8519cfb-620b-4c16-8ce7-43587912842e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:45.717680 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8519cfb-620b-4c16-8ce7-43587912842e admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:45.717954 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8519cfb-620b-4c16-8ce7-43587912842e admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:45.718252 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8519cfb-620b-4c16-8ce7-43587912842e admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:45.718946 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8519cfb-620b-4c16-8ce7-43587912842e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:45.720553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8519cfb-620b-4c16-8ce7-43587912842e admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:45.732358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1388/2777] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:02:45.736771 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f6d3e-ee6a-46e0-8873-b22f5d69acde admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:45.747682 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f6d3e-ee6a-46e0-8873-b22f5d69acde admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:45.752239 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4255754-83e2-44eb-b40d-adc7f8ef34b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:45.752881 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4255754-83e2-44eb-b40d-adc7f8ef34b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:45.753200 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4255754-83e2-44eb-b40d-adc7f8ef34b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:45.753472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4255754-83e2-44eb-b40d-adc7f8ef34b8 admin admin] PATH_INFO: `/v3/projects/c52ebf9bce564decb9369f8c56ebed4f/users/2544c5a915ca4b45a7c29a2e922f76b2/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:45.768850 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4255754-83e2-44eb-b40d-adc7f8ef34b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c52ebf9bce564decb9369f8c56ebed4f, user_id=2544c5a915ca4b45a7c29a2e922f76b2, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:45.772623 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4255754-83e2-44eb-b40d-adc7f8ef34b8 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:45.834527 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1390/2778] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:02:45 2022] PUT /identity/v3/projects/c52ebf9bce564decb9369f8c56ebed4f/users/2544c5a915ca4b45a7c29a2e922f76b2/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:45.838485 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8519cfb-620b-4c16-8ce7-43587912842e admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:45.848549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8519cfb-620b-4c16-8ce7-43587912842e admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:45.903977 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e62fc2-dca2-4b9c-bc8d-9a7dc69d113f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:45.904558 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e62fc2-dca2-4b9c-bc8d-9a7dc69d113f admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:45.904830 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e62fc2-dca2-4b9c-bc8d-9a7dc69d113f admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:45.905095 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e62fc2-dca2-4b9c-bc8d-9a7dc69d113f admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:45.905714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e62fc2-dca2-4b9c-bc8d-9a7dc69d113f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:45.907324 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e62fc2-dca2-4b9c-bc8d-9a7dc69d113f admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:45.917918 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1389/2779] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:02:45.922148 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4255754-83e2-44eb-b40d-adc7f8ef34b8 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:45.933012 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4255754-83e2-44eb-b40d-adc7f8ef34b8 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:45.937901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9bd9d-e746-45a6-87b4-db3b33ce4c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:45.938561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9bd9d-e746-45a6-87b4-db3b33ce4c82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:45.938871 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9bd9d-e746-45a6-87b4-db3b33ce4c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:45.939142 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9bd9d-e746-45a6-87b4-db3b33ce4c82 admin admin] PATH_INFO: `/v3/projects/c52ebf9bce564decb9369f8c56ebed4f/users/2544c5a915ca4b45a7c29a2e922f76b2/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:45.941108 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a9bd9d-e746-45a6-87b4-db3b33ce4c82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c52ebf9bce564decb9369f8c56ebed4f, user_id=2544c5a915ca4b45a7c29a2e922f76b2, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:45.943977 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a9bd9d-e746-45a6-87b4-db3b33ce4c82 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:45.954163 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1391/2780] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:02:45 2022] PUT /identity/v3/projects/c52ebf9bce564decb9369f8c56ebed4f/users/2544c5a915ca4b45a7c29a2e922f76b2/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:02:45.964872 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d035f2-3cc8-41d4-abce-86ff3f87ad8c None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:45.965160 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d035f2-3cc8-41d4-abce-86ff3f87ad8c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:45.965431 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d035f2-3cc8-41d4-abce-86ff3f87ad8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:45.994158 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-d6d035f2-3cc8-41d4-abce-86ff3f87ad8c None None] MFA Rules not processed for user `2544c5a915ca4b45a7c29a2e922f76b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:45.995399 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d6d035f2-3cc8-41d4-abce-86ff3f87ad8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:46.062375 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1390/2781] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:02:45 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:46.074139 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9bd9d-e746-45a6-87b4-db3b33ce4c82 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:46.075840 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-19a9bd9d-e746-45a6-87b4-db3b33ce4c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:46.082399 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:46.135766 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9bd9d-e746-45a6-87b4-db3b33ce4c82 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:46.139404 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f909cd-df62-4062-929d-f182e9453eec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:46.140056 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f909cd-df62-4062-929d-f182e9453eec service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:46.140357 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f909cd-df62-4062-929d-f182e9453eec service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:46.140625 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f909cd-df62-4062-929d-f182e9453eec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:46.151222 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f909cd-df62-4062-929d-f182e9453eec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:46.152727 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f909cd-df62-4062-929d-f182e9453eec service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:46.162669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1392/2782] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:46 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:02:46.231754 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d035f2-3cc8-41d4-abce-86ff3f87ad8c None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:46.294971 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d035f2-3cc8-41d4-abce-86ff3f87ad8c None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:46.299533 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d1f60-4e11-4c15-ab01-f9f35b77904a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:46.300154 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d1f60-4e11-4c15-ab01-f9f35b77904a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:46.300430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d1f60-4e11-4c15-ab01-f9f35b77904a admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:46.300696 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d1f60-4e11-4c15-ab01-f9f35b77904a admin admin] PATH_INFO: `/v3/users/9aea4e5113184f798b8f1e9cc7bf4b55` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:46.301842 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d1f60-4e11-4c15-ab01-f9f35b77904a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aea4e5113184f798b8f1e9cc7bf4b55)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:46.304136 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d1f60-4e11-4c15-ab01-f9f35b77904a admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:46.315998 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:46.494800 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-175d1f60-4e11-4c15-ab01-f9f35b77904a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aea4e5113184f798b8f1e9cc7bf4b55'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:46.607583 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-175d1f60-4e11-4c15-ab01-f9f35b77904a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aea4e5113184f798b8f1e9cc7bf4b55'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:46.608192 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-175d1f60-4e11-4c15-ab01-f9f35b77904a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aea4e5113184f798b8f1e9cc7bf4b55'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:46.616432 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-175d1f60-4e11-4c15-ab01-f9f35b77904a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aea4e5113184f798b8f1e9cc7bf4b55'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:46.617064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-175d1f60-4e11-4c15-ab01-f9f35b77904a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aea4e5113184f798b8f1e9cc7bf4b55'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:46.635302 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1391/2783] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:02:46 2022] DELETE /identity/v3/users/9aea4e5113184f798b8f1e9cc7bf4b55 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:46.639830 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2b51d-30b2-4591-83a2-407e99cad5f1 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:46.640296 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2b51d-30b2-4591-83a2-407e99cad5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:46.640394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2b51d-30b2-4591-83a2-407e99cad5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:46.665313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-11a2b51d-30b2-4591-83a2-407e99cad5f1 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:46.666531 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-11a2b51d-30b2-4591-83a2-407e99cad5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:46.726179 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1393/2784] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:02:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:46.736187 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d1f60-4e11-4c15-ab01-f9f35b77904a admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:46.737609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-175d1f60-4e11-4c15-ab01-f9f35b77904a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:46.796991 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d1f60-4e11-4c15-ab01-f9f35b77904a admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:46.800633 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e6f07-efe1-4615-9de4-0fb2adc6b7cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:46.801233 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e6f07-efe1-4615-9de4-0fb2adc6b7cf service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:46.801509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e6f07-efe1-4615-9de4-0fb2adc6b7cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:46.801776 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e6f07-efe1-4615-9de4-0fb2adc6b7cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:46.812522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e6f07-efe1-4615-9de4-0fb2adc6b7cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:46.814121 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e6f07-efe1-4615-9de4-0fb2adc6b7cf service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:46.824048 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1392/2785] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:02:47.241575 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2b51d-30b2-4591-83a2-407e99cad5f1 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:47.243139 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-11a2b51d-30b2-4591-83a2-407e99cad5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:47.259374 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2b51d-30b2-4591-83a2-407e99cad5f1 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:47.264848 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7aa0bf-75c8-4eee-87f1-3a8312b4ab0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:47.265817 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7aa0bf-75c8-4eee-87f1-3a8312b4ab0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:47.266213 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7aa0bf-75c8-4eee-87f1-3a8312b4ab0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:47.266371 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7aa0bf-75c8-4eee-87f1-3a8312b4ab0f admin admin] PATH_INFO: `/v3/projects/55d9a4c7ea23452eb70340774386e071` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:47.267581 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7aa0bf-75c8-4eee-87f1-3a8312b4ab0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=55d9a4c7ea23452eb70340774386e071)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:47.270217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7aa0bf-75c8-4eee-87f1-3a8312b4ab0f admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:47.385566 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5d7aa0bf-75c8-4eee-87f1-3a8312b4ab0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55d9a4c7ea23452eb70340774386e071', 'request_id': 'req-5d7aa0bf-75c8-4eee-87f1-3a8312b4ab0f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:47.470576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5d7aa0bf-75c8-4eee-87f1-3a8312b4ab0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55d9a4c7ea23452eb70340774386e071', 'request_id': 'req-5d7aa0bf-75c8-4eee-87f1-3a8312b4ab0f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:47.478353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5d7aa0bf-75c8-4eee-87f1-3a8312b4ab0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55d9a4c7ea23452eb70340774386e071', 'request_id': 'req-5d7aa0bf-75c8-4eee-87f1-3a8312b4ab0f'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:47.488783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5d7aa0bf-75c8-4eee-87f1-3a8312b4ab0f admin admin] The token cache is being invalidate because project 55d9a4c7ea23452eb70340774386e071 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:02:47.489064 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5d7aa0bf-75c8-4eee-87f1-3a8312b4ab0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:47.489710 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5d7aa0bf-75c8-4eee-87f1-3a8312b4ab0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:47.491346 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1394/2786] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:02:47 2022] DELETE /identity/v3/projects/55d9a4c7ea23452eb70340774386e071 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:47.495487 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e6f07-efe1-4615-9de4-0fb2adc6b7cf service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:47.505659 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e6f07-efe1-4615-9de4-0fb2adc6b7cf service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:47.561727 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d954c3-c4e0-4d32-bdb9-fbfcaaece647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:47.562347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d954c3-c4e0-4d32-bdb9-fbfcaaece647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:47.562665 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d954c3-c4e0-4d32-bdb9-fbfcaaece647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:47.562941 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d954c3-c4e0-4d32-bdb9-fbfcaaece647 admin admin] PATH_INFO: `/v3/users/2544c5a915ca4b45a7c29a2e922f76b2` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:47.564045 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d954c3-c4e0-4d32-bdb9-fbfcaaece647 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2544c5a915ca4b45a7c29a2e922f76b2)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:47.565992 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d954c3-c4e0-4d32-bdb9-fbfcaaece647 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:47.727756 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:47.740694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-32d954c3-c4e0-4d32-bdb9-fbfcaaece647 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2544c5a915ca4b45a7c29a2e922f76b2'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:47.824758 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-32d954c3-c4e0-4d32-bdb9-fbfcaaece647 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2544c5a915ca4b45a7c29a2e922f76b2'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:47.826079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-32d954c3-c4e0-4d32-bdb9-fbfcaaece647 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2544c5a915ca4b45a7c29a2e922f76b2'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:47.834578 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-32d954c3-c4e0-4d32-bdb9-fbfcaaece647 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2544c5a915ca4b45a7c29a2e922f76b2'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:47.835719 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-32d954c3-c4e0-4d32-bdb9-fbfcaaece647 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2544c5a915ca4b45a7c29a2e922f76b2'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:47.852092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1393/2787] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:02:47 2022] DELETE /identity/v3/users/2544c5a915ca4b45a7c29a2e922f76b2 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:48.321473 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7aa0bf-75c8-4eee-87f1-3a8312b4ab0f admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:48.323009 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-5d7aa0bf-75c8-4eee-87f1-3a8312b4ab0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:48.336780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7aa0bf-75c8-4eee-87f1-3a8312b4ab0f admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:48.368202 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:48.396644 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4a8b0-fa26-4e76-bc8d-5589eefeef37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:48.397278 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4a8b0-fa26-4e76-bc8d-5589eefeef37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:48.397696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4a8b0-fa26-4e76-bc8d-5589eefeef37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:48.397972 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4a8b0-fa26-4e76-bc8d-5589eefeef37 admin admin] PATH_INFO: `/v3/projects/c52ebf9bce564decb9369f8c56ebed4f` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:48.399167 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4a8b0-fa26-4e76-bc8d-5589eefeef37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c52ebf9bce564decb9369f8c56ebed4f)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:48.401075 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4a8b0-fa26-4e76-bc8d-5589eefeef37 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:48.604133 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-42a4a8b0-fa26-4e76-bc8d-5589eefeef37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c52ebf9bce564decb9369f8c56ebed4f', 'request_id': 'req-42a4a8b0-fa26-4e76-bc8d-5589eefeef37'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:48.698697 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-42a4a8b0-fa26-4e76-bc8d-5589eefeef37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c52ebf9bce564decb9369f8c56ebed4f', 'request_id': 'req-42a4a8b0-fa26-4e76-bc8d-5589eefeef37'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:48.705557 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-42a4a8b0-fa26-4e76-bc8d-5589eefeef37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c52ebf9bce564decb9369f8c56ebed4f', 'request_id': 'req-42a4a8b0-fa26-4e76-bc8d-5589eefeef37'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:48.720074 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-42a4a8b0-fa26-4e76-bc8d-5589eefeef37 admin admin] The token cache is being invalidate because project c52ebf9bce564decb9369f8c56ebed4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:02:48.720074 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-42a4a8b0-fa26-4e76-bc8d-5589eefeef37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:48.721087 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-42a4a8b0-fa26-4e76-bc8d-5589eefeef37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:48.722280 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1395/2788] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:02:48 2022] DELETE /identity/v3/projects/c52ebf9bce564decb9369f8c56ebed4f => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:48.728241 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecded61-c0de-4849-b405-b9ec9a7664c1 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:48.728762 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecded61-c0de-4849-b405-b9ec9a7664c1 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:48.728863 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecded61-c0de-4849-b405-b9ec9a7664c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:48.750133 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-6ecded61-c0de-4849-b405-b9ec9a7664c1 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:48.751401 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-6ecded61-c0de-4849-b405-b9ec9a7664c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:48.812810 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1394/2789] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:02:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:48.821597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c0817-fe70-4ed8-ab43-24f4b9401317 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:48.821880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c0817-fe70-4ed8-ab43-24f4b9401317 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:48.822155 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c0817-fe70-4ed8-ab43-24f4b9401317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:48.842545 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-6a9c0817-fe70-4ed8-ab43-24f4b9401317 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:48.843776 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-6a9c0817-fe70-4ed8-ab43-24f4b9401317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:48.852766 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1396/2790] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:02:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:48.857111 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecded61-c0de-4849-b405-b9ec9a7664c1 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:48.867733 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecded61-c0de-4849-b405-b9ec9a7664c1 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:48.874327 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926d13c-1450-446b-8b5e-6ec019d3dd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:48.875005 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926d13c-1450-446b-8b5e-6ec019d3dd28 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:48.875297 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926d13c-1450-446b-8b5e-6ec019d3dd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:48.875565 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926d13c-1450-446b-8b5e-6ec019d3dd28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:48.876212 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926d13c-1450-446b-8b5e-6ec019d3dd28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:02:48.876510 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926d13c-1450-446b-8b5e-6ec019d3dd28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:48.878125 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926d13c-1450-446b-8b5e-6ec019d3dd28 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:48.890163 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1395/2791] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 17:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:02:48.894533 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c0817-fe70-4ed8-ab43-24f4b9401317 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:48.904414 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c0817-fe70-4ed8-ab43-24f4b9401317 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:48.909116 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e04065-f17a-4e91-aa14-af17e1e1230c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:48.909710 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e04065-f17a-4e91-aa14-af17e1e1230c admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:48.909984 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e04065-f17a-4e91-aa14-af17e1e1230c admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:48.910246 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e04065-f17a-4e91-aa14-af17e1e1230c admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:48.910905 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e04065-f17a-4e91-aa14-af17e1e1230c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:48.913677 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e04065-f17a-4e91-aa14-af17e1e1230c admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:49.032217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1397/2792] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:02:48 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:02:49.036625 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926d13c-1450-446b-8b5e-6ec019d3dd28 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:49.046637 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926d13c-1450-446b-8b5e-6ec019d3dd28 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:49.104301 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce9774-7067-4190-925e-6f085c8f85c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:49.104909 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce9774-7067-4190-925e-6f085c8f85c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:49.105186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce9774-7067-4190-925e-6f085c8f85c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:49.107436 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce9774-7067-4190-925e-6f085c8f85c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:49.107436 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce9774-7067-4190-925e-6f085c8f85c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:49.108069 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce9774-7067-4190-925e-6f085c8f85c4 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:49.195643 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1396/2793] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:02:49 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:02:49.200494 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e04065-f17a-4e91-aa14-af17e1e1230c admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:49.212253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e04065-f17a-4e91-aa14-af17e1e1230c admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:49.216922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497dd49a-c201-4724-bea8-0f72c423db3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:49.217522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dd49a-c201-4724-bea8-0f72c423db3f admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:49.217798 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dd49a-c201-4724-bea8-0f72c423db3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:49.218093 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dd49a-c201-4724-bea8-0f72c423db3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:49.218774 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497dd49a-c201-4724-bea8-0f72c423db3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:49.220360 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497dd49a-c201-4724-bea8-0f72c423db3f admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:49.230870 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1398/2794] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:02:49.235320 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce9774-7067-4190-925e-6f085c8f85c4 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:49.246685 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce9774-7067-4190-925e-6f085c8f85c4 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:49.251396 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f1da4-7007-4769-95aa-bba673247086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:49.252079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f1da4-7007-4769-95aa-bba673247086 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:49.252348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f1da4-7007-4769-95aa-bba673247086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:49.252613 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f1da4-7007-4769-95aa-bba673247086 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:49.253248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f1da4-7007-4769-95aa-bba673247086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:49.254865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f1da4-7007-4769-95aa-bba673247086 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:49.267430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1397/2795] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:02:49.272264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497dd49a-c201-4724-bea8-0f72c423db3f admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:49.282820 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497dd49a-c201-4724-bea8-0f72c423db3f admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:49.288106 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e0d52-816b-42f9-9a91-922de95876dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:49.288794 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e0d52-816b-42f9-9a91-922de95876dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:49.289064 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e0d52-816b-42f9-9a91-922de95876dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:49.289356 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e0d52-816b-42f9-9a91-922de95876dc admin admin] PATH_INFO: `/v3/projects/ef1412f0ae4e406c900cba298b88755b/users/b07f20cf93754dd19b3c18b71534734b/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:49.304452 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e0d52-816b-42f9-9a91-922de95876dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef1412f0ae4e406c900cba298b88755b, user_id=b07f20cf93754dd19b3c18b71534734b, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:49.307133 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e0d52-816b-42f9-9a91-922de95876dc admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:49.386191 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1399/2796] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:02:49 2022] PUT /identity/v3/projects/ef1412f0ae4e406c900cba298b88755b/users/b07f20cf93754dd19b3c18b71534734b/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:49.397225 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd8dc4-beca-4e7b-9f64-e28ea7d2bcc7 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:49.397533 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd8dc4-beca-4e7b-9f64-e28ea7d2bcc7 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:49.397807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd8dc4-beca-4e7b-9f64-e28ea7d2bcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:49.427882 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-d0cd8dc4-beca-4e7b-9f64-e28ea7d2bcc7 None None] MFA Rules not processed for user `b07f20cf93754dd19b3c18b71534734b`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:49.429111 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d0cd8dc4-beca-4e7b-9f64-e28ea7d2bcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:49.433207 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:49.501781 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1398/2797] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:02:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:49.507158 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e0d52-816b-42f9-9a91-922de95876dc admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:49.567427 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:49.571950 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e0d52-816b-42f9-9a91-922de95876dc admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:49.576632 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51744144-ff40-4f6d-9935-6cfd21edcc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:49.577243 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51744144-ff40-4f6d-9935-6cfd21edcc48 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:49.581504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51744144-ff40-4f6d-9935-6cfd21edcc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:49.581504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51744144-ff40-4f6d-9935-6cfd21edcc48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:49.581504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51744144-ff40-4f6d-9935-6cfd21edcc48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:49.581504 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51744144-ff40-4f6d-9935-6cfd21edcc48 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:49.649413 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1400/2798] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:02:49 2022] POST /identity/v3/projects => generated 387 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:02:49.653484 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd8dc4-beca-4e7b-9f64-e28ea7d2bcc7 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:49.665181 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd8dc4-beca-4e7b-9f64-e28ea7d2bcc7 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:49.729596 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02146b-bb6e-4fd6-a542-3ed42f32c3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:49.785861 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02146b-bb6e-4fd6-a542-3ed42f32c3e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:49.785861 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02146b-bb6e-4fd6-a542-3ed42f32c3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:49.785861 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02146b-bb6e-4fd6-a542-3ed42f32c3e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:49.785861 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea02146b-bb6e-4fd6-a542-3ed42f32c3e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:49.785861 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea02146b-bb6e-4fd6-a542-3ed42f32c3e6 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:49.943332 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1399/2799] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:02:49 2022] POST /identity/v3/users => generated 294 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:02:49.947247 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51744144-ff40-4f6d-9935-6cfd21edcc48 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:49.957832 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51744144-ff40-4f6d-9935-6cfd21edcc48 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:49.963369 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033dbf8c-76f3-4d55-9b54-26815621f632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:49.963965 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033dbf8c-76f3-4d55-9b54-26815621f632 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:49.964240 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033dbf8c-76f3-4d55-9b54-26815621f632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:49.964507 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033dbf8c-76f3-4d55-9b54-26815621f632 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:49.965137 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033dbf8c-76f3-4d55-9b54-26815621f632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:49.966809 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033dbf8c-76f3-4d55-9b54-26815621f632 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:49.977120 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1401/2800] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:02:49.981175 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02146b-bb6e-4fd6-a542-3ed42f32c3e6 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:49.991466 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02146b-bb6e-4fd6-a542-3ed42f32c3e6 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:49.996055 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7bd74c-9e89-42dc-b15e-7d2828423b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:49.996632 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7bd74c-9e89-42dc-b15e-7d2828423b63 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:49.996914 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7bd74c-9e89-42dc-b15e-7d2828423b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:49.997185 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7bd74c-9e89-42dc-b15e-7d2828423b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:49.997820 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7bd74c-9e89-42dc-b15e-7d2828423b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:49.999452 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7bd74c-9e89-42dc-b15e-7d2828423b63 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:50.010031 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1400/2801] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:02:50.014181 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033dbf8c-76f3-4d55-9b54-26815621f632 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:50.026912 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033dbf8c-76f3-4d55-9b54-26815621f632 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:50.031469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b67f37-ab38-4e42-9caa-131307a62a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:50.032117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b67f37-ab38-4e42-9caa-131307a62a71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:50.032394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b67f37-ab38-4e42-9caa-131307a62a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:50.032662 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b67f37-ab38-4e42-9caa-131307a62a71 admin admin] PATH_INFO: `/v3/projects/bd1f36560093430094534e3bb8bde698/users/07ad24209be742a7a5b23d53a3fa24d3/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:50.047089 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b67f37-ab38-4e42-9caa-131307a62a71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd1f36560093430094534e3bb8bde698, user_id=07ad24209be742a7a5b23d53a3fa24d3, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:50.049691 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b67f37-ab38-4e42-9caa-131307a62a71 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:50.117222 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1402/2802] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:02:50 2022] PUT /identity/v3/projects/bd1f36560093430094534e3bb8bde698/users/07ad24209be742a7a5b23d53a3fa24d3/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:02:50.127766 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92778ac-6a4d-42f5-868e-12ca3fdbb24c None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:50.128056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92778ac-6a4d-42f5-868e-12ca3fdbb24c None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:50.128358 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92778ac-6a4d-42f5-868e-12ca3fdbb24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:50.157294 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-d92778ac-6a4d-42f5-868e-12ca3fdbb24c None None] MFA Rules not processed for user `07ad24209be742a7a5b23d53a3fa24d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:50.158512 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d92778ac-6a4d-42f5-868e-12ca3fdbb24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:50.222864 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1401/2803] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:02:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:50.227463 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b67f37-ab38-4e42-9caa-131307a62a71 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:50.289766 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b67f37-ab38-4e42-9caa-131307a62a71 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:50.294387 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1419a-a0f9-40b6-8d03-00bf25dbfd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:50.295020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1419a-a0f9-40b6-8d03-00bf25dbfd49 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:50.295297 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1419a-a0f9-40b6-8d03-00bf25dbfd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:50.295563 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1419a-a0f9-40b6-8d03-00bf25dbfd49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:50.296179 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa1419a-a0f9-40b6-8d03-00bf25dbfd49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:50.298055 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa1419a-a0f9-40b6-8d03-00bf25dbfd49 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:50.403311 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1403/2804] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:02:50 2022] POST /identity/v3/projects => generated 387 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:02:50.407537 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92778ac-6a4d-42f5-868e-12ca3fdbb24c None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:50.418507 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92778ac-6a4d-42f5-868e-12ca3fdbb24c None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:50.427704 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:50.476082 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520dbe9-946b-494d-ada9-851334a91b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:50.476707 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520dbe9-946b-494d-ada9-851334a91b35 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:50.476930 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520dbe9-946b-494d-ada9-851334a91b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:50.477203 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520dbe9-946b-494d-ada9-851334a91b35 admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:50.477975 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520dbe9-946b-494d-ada9-851334a91b35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:50.480392 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520dbe9-946b-494d-ada9-851334a91b35 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:50.562209 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1402/2805] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:02:50 2022] POST /identity/v3/users => generated 300 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:02:50.566236 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1419a-a0f9-40b6-8d03-00bf25dbfd49 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:50.576522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1419a-a0f9-40b6-8d03-00bf25dbfd49 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:50.581603 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417ff09-4c38-4a2b-b89e-70dab0851176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:50.582209 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417ff09-4c38-4a2b-b89e-70dab0851176 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:50.582515 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417ff09-4c38-4a2b-b89e-70dab0851176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:50.582843 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417ff09-4c38-4a2b-b89e-70dab0851176 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:50.583489 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417ff09-4c38-4a2b-b89e-70dab0851176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:50.585472 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417ff09-4c38-4a2b-b89e-70dab0851176 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:50.596076 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1404/2806] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:02:50.600668 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520dbe9-946b-494d-ada9-851334a91b35 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:50.610542 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520dbe9-946b-494d-ada9-851334a91b35 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:50.616096 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba0b17-412b-4745-bf79-dd62b500a56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:50.616746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba0b17-412b-4745-bf79-dd62b500a56c admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:50.617019 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba0b17-412b-4745-bf79-dd62b500a56c admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:50.617287 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba0b17-412b-4745-bf79-dd62b500a56c admin admin] PATH_INFO: `/v3/projects/f1e12c012fd6420d8e049ba6f5de2aff/users/4726a0fcd9134abda9cf1f73958e2235/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:50.632208 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba0b17-412b-4745-bf79-dd62b500a56c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1e12c012fd6420d8e049ba6f5de2aff, user_id=4726a0fcd9134abda9cf1f73958e2235, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:50.634895 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba0b17-412b-4745-bf79-dd62b500a56c admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:50.691404 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1403/2807] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:02:50 2022] PUT /identity/v3/projects/f1e12c012fd6420d8e049ba6f5de2aff/users/4726a0fcd9134abda9cf1f73958e2235/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:50.695522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417ff09-4c38-4a2b-b89e-70dab0851176 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:50.705637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417ff09-4c38-4a2b-b89e-70dab0851176 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:50.765383 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47838e-b532-49b3-8027-6686a43aea8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:50.765905 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47838e-b532-49b3-8027-6686a43aea8c admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:50.766185 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47838e-b532-49b3-8027-6686a43aea8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:50.766450 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47838e-b532-49b3-8027-6686a43aea8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:50.767152 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47838e-b532-49b3-8027-6686a43aea8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:50.768996 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47838e-b532-49b3-8027-6686a43aea8c admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:50.779783 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1405/2808] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:02:50.784351 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba0b17-412b-4745-bf79-dd62b500a56c admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:50.796708 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba0b17-412b-4745-bf79-dd62b500a56c admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:50.801757 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa77315-972a-4594-b66d-8994b8e52bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:50.803115 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa77315-972a-4594-b66d-8994b8e52bac admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:50.803413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa77315-972a-4594-b66d-8994b8e52bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:50.803705 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa77315-972a-4594-b66d-8994b8e52bac admin admin] PATH_INFO: `/v3/projects/f1e12c012fd6420d8e049ba6f5de2aff/users/4726a0fcd9134abda9cf1f73958e2235/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:50.805729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa77315-972a-4594-b66d-8994b8e52bac admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1e12c012fd6420d8e049ba6f5de2aff, user_id=4726a0fcd9134abda9cf1f73958e2235, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:50.808553 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa77315-972a-4594-b66d-8994b8e52bac admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:50.819675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1404/2809] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:02:50 2022] PUT /identity/v3/projects/f1e12c012fd6420d8e049ba6f5de2aff/users/4726a0fcd9134abda9cf1f73958e2235/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:50.830953 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdee21-ec2f-4210-befb-ace99ee06d3a None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:50.831232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdee21-ec2f-4210-befb-ace99ee06d3a None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:50.831527 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdee21-ec2f-4210-befb-ace99ee06d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:50.859308 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-f0fdee21-ec2f-4210-befb-ace99ee06d3a None None] MFA Rules not processed for user `4726a0fcd9134abda9cf1f73958e2235`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:50.860548 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f0fdee21-ec2f-4210-befb-ace99ee06d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:50.866196 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:50.930038 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1406/2810] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:02:50 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:50.941463 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa77315-972a-4594-b66d-8994b8e52bac admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:50.944245 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-0fa77315-972a-4594-b66d-8994b8e52bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:51.001838 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa77315-972a-4594-b66d-8994b8e52bac admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:51.006092 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27737593-8336-41f2-a658-744fa0a91ef2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:51.006806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27737593-8336-41f2-a658-744fa0a91ef2 service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:51.007083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27737593-8336-41f2-a658-744fa0a91ef2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:51.007344 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27737593-8336-41f2-a658-744fa0a91ef2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:51.016519 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27737593-8336-41f2-a658-744fa0a91ef2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:51.017922 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27737593-8336-41f2-a658-744fa0a91ef2 service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:51.027608 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1405/2811] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:02:52.088683 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdee21-ec2f-4210-befb-ace99ee06d3a None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:52.090100 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f0fdee21-ec2f-4210-befb-ace99ee06d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:52.103427 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdee21-ec2f-4210-befb-ace99ee06d3a None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:52.107017 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d69b58-c8e6-45a6-8d4f-043bbe0a5250 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:52.107627 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d69b58-c8e6-45a6-8d4f-043bbe0a5250 service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:52.107902 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d69b58-c8e6-45a6-8d4f-043bbe0a5250 service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:52.108264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d69b58-c8e6-45a6-8d4f-043bbe0a5250 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:52.167101 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d69b58-c8e6-45a6-8d4f-043bbe0a5250 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:52.168527 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d69b58-c8e6-45a6-8d4f-043bbe0a5250 service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:52.178247 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1407/2812] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:02:57.198488 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27737593-8336-41f2-a658-744fa0a91ef2 service cinder] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:57.204657 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27737593-8336-41f2-a658-744fa0a91ef2 service cinder] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:57.208370 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e793f-25bf-462e-a4a7-559bde0835b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:57.209033 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e793f-25bf-462e-a4a7-559bde0835b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:57.209307 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e793f-25bf-462e-a4a7-559bde0835b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:57.209570 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e793f-25bf-462e-a4a7-559bde0835b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:57.271435 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e793f-25bf-462e-a4a7-559bde0835b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:57.272880 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e793f-25bf-462e-a4a7-559bde0835b5 service cinder] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:57.283268 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1406/2813] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:57 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:03:01.354636 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d69b58-c8e6-45a6-8d4f-043bbe0a5250 service cinder] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:01.417561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d69b58-c8e6-45a6-8d4f-043bbe0a5250 service cinder] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:01.422097 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b11da3-0d9b-4bc8-9809-df9426cfef9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:01.422789 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b11da3-0d9b-4bc8-9809-df9426cfef9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:01.423089 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b11da3-0d9b-4bc8-9809-df9426cfef9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:01.423343 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b11da3-0d9b-4bc8-9809-df9426cfef9a admin admin] PATH_INFO: `/v3/users/b07f20cf93754dd19b3c18b71534734b` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:01.424479 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b11da3-0d9b-4bc8-9809-df9426cfef9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b07f20cf93754dd19b3c18b71534734b)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:01.426395 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b11da3-0d9b-4bc8-9809-df9426cfef9a admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:01.611705 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:01.614364 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-42b11da3-0d9b-4bc8-9809-df9426cfef9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b07f20cf93754dd19b3c18b71534734b'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:01.707937 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-42b11da3-0d9b-4bc8-9809-df9426cfef9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b07f20cf93754dd19b3c18b71534734b'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:01.708551 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-42b11da3-0d9b-4bc8-9809-df9426cfef9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b07f20cf93754dd19b3c18b71534734b'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:01.717520 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-42b11da3-0d9b-4bc8-9809-df9426cfef9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b07f20cf93754dd19b3c18b71534734b'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:01.731172 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-42b11da3-0d9b-4bc8-9809-df9426cfef9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b07f20cf93754dd19b3c18b71534734b'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:01.737144 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1408/2814] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:03:01 2022] DELETE /identity/v3/users/b07f20cf93754dd19b3c18b71534734b => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:03:01.741633 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c313454-6549-4bd6-923d-f3a1dca1bca2 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:01.742154 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c313454-6549-4bd6-923d-f3a1dca1bca2 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:01.742154 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c313454-6549-4bd6-923d-f3a1dca1bca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:01.755226 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:01.761476 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-5c313454-6549-4bd6-923d-f3a1dca1bca2 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:01.765423 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-5c313454-6549-4bd6-923d-f3a1dca1bca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:01.825100 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1407/2815] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:01.834992 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b11da3-0d9b-4bc8-9809-df9426cfef9a admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:01.836433 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-42b11da3-0d9b-4bc8-9809-df9426cfef9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:01.894477 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b11da3-0d9b-4bc8-9809-df9426cfef9a admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:01.898057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a451877-354a-4515-b302-8c333e45980d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:01.898696 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a451877-354a-4515-b302-8c333e45980d service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:01.898969 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a451877-354a-4515-b302-8c333e45980d service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:01.899232 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a451877-354a-4515-b302-8c333e45980d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:01.909175 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a451877-354a-4515-b302-8c333e45980d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:01.910611 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a451877-354a-4515-b302-8c333e45980d service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:01.920480 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1409/2816] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:03:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:03:02.300764 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c313454-6549-4bd6-923d-f3a1dca1bca2 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:02.302214 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-5c313454-6549-4bd6-923d-f3a1dca1bca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:02.319074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c313454-6549-4bd6-923d-f3a1dca1bca2 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:02.326146 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689857f-6bc5-4d4d-b4fd-0c8ca41a83a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:02.326928 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689857f-6bc5-4d4d-b4fd-0c8ca41a83a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:02.327074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689857f-6bc5-4d4d-b4fd-0c8ca41a83a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:02.327341 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689857f-6bc5-4d4d-b4fd-0c8ca41a83a0 admin admin] PATH_INFO: `/v3/projects/ef1412f0ae4e406c900cba298b88755b` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:02.328524 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689857f-6bc5-4d4d-b4fd-0c8ca41a83a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef1412f0ae4e406c900cba298b88755b)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:02.330384 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689857f-6bc5-4d4d-b4fd-0c8ca41a83a0 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:02.455621 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e689857f-6bc5-4d4d-b4fd-0c8ca41a83a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef1412f0ae4e406c900cba298b88755b', 'request_id': 'req-e689857f-6bc5-4d4d-b4fd-0c8ca41a83a0'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:02.461290 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e689857f-6bc5-4d4d-b4fd-0c8ca41a83a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef1412f0ae4e406c900cba298b88755b', 'request_id': 'req-e689857f-6bc5-4d4d-b4fd-0c8ca41a83a0'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:02.468549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e689857f-6bc5-4d4d-b4fd-0c8ca41a83a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef1412f0ae4e406c900cba298b88755b', 'request_id': 'req-e689857f-6bc5-4d4d-b4fd-0c8ca41a83a0'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:02.542174 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e689857f-6bc5-4d4d-b4fd-0c8ca41a83a0 admin admin] The token cache is being invalidate because project ef1412f0ae4e406c900cba298b88755b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:03:02.542459 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e689857f-6bc5-4d4d-b4fd-0c8ca41a83a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:02.543104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-e689857f-6bc5-4d4d-b4fd-0c8ca41a83a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:02.544629 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1408/2817] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:03:02 2022] DELETE /identity/v3/projects/ef1412f0ae4e406c900cba298b88755b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:03:02.548694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a451877-354a-4515-b302-8c333e45980d service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:02.558940 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a451877-354a-4515-b302-8c333e45980d service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:02.614668 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8d1a5-0f98-4cc5-95a1-f97737bbbca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:02.615305 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8d1a5-0f98-4cc5-95a1-f97737bbbca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:02.615581 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8d1a5-0f98-4cc5-95a1-f97737bbbca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:02.615852 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8d1a5-0f98-4cc5-95a1-f97737bbbca7 admin admin] PATH_INFO: `/v3/users/07ad24209be742a7a5b23d53a3fa24d3` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:02.617029 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a8d1a5-0f98-4cc5-95a1-f97737bbbca7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07ad24209be742a7a5b23d53a3fa24d3)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:02.618981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a8d1a5-0f98-4cc5-95a1-f97737bbbca7 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:02.850234 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-e1a8d1a5-0f98-4cc5-95a1-f97737bbbca7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07ad24209be742a7a5b23d53a3fa24d3'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:02.916185 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-e1a8d1a5-0f98-4cc5-95a1-f97737bbbca7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07ad24209be742a7a5b23d53a3fa24d3'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:02.917075 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-e1a8d1a5-0f98-4cc5-95a1-f97737bbbca7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07ad24209be742a7a5b23d53a3fa24d3'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:02.925397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-e1a8d1a5-0f98-4cc5-95a1-f97737bbbca7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07ad24209be742a7a5b23d53a3fa24d3'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:02.927793 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:02.938993 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-e1a8d1a5-0f98-4cc5-95a1-f97737bbbca7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07ad24209be742a7a5b23d53a3fa24d3'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:02.944804 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1410/2818] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:03:02 2022] DELETE /identity/v3/users/07ad24209be742a7a5b23d53a3fa24d3 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:03:03.361927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689857f-6bc5-4d4d-b4fd-0c8ca41a83a0 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:03.363406 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-e689857f-6bc5-4d4d-b4fd-0c8ca41a83a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:03.378450 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689857f-6bc5-4d4d-b4fd-0c8ca41a83a0 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:03.441703 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc22444-12d6-4078-95d4-317b3d420201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:03.442340 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc22444-12d6-4078-95d4-317b3d420201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:03.442644 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc22444-12d6-4078-95d4-317b3d420201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:03.442916 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc22444-12d6-4078-95d4-317b3d420201 admin admin] PATH_INFO: `/v3/projects/bd1f36560093430094534e3bb8bde698` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:03.444087 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc22444-12d6-4078-95d4-317b3d420201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd1f36560093430094534e3bb8bde698)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:03.445962 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc22444-12d6-4078-95d4-317b3d420201 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:03.528372 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:03.541820 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-fcc22444-12d6-4078-95d4-317b3d420201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd1f36560093430094534e3bb8bde698', 'request_id': 'req-fcc22444-12d6-4078-95d4-317b3d420201'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:03.547317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-fcc22444-12d6-4078-95d4-317b3d420201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd1f36560093430094534e3bb8bde698', 'request_id': 'req-fcc22444-12d6-4078-95d4-317b3d420201'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:03.553555 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-fcc22444-12d6-4078-95d4-317b3d420201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd1f36560093430094534e3bb8bde698', 'request_id': 'req-fcc22444-12d6-4078-95d4-317b3d420201'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:03.709818 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-fcc22444-12d6-4078-95d4-317b3d420201 admin admin] The token cache is being invalidate because project bd1f36560093430094534e3bb8bde698 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:03:03.710085 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-fcc22444-12d6-4078-95d4-317b3d420201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:03.710778 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-fcc22444-12d6-4078-95d4-317b3d420201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:03.712342 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1409/2819] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:03:03 2022] DELETE /identity/v3/projects/bd1f36560093430094534e3bb8bde698 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:03.717882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8d1a5-0f98-4cc5-95a1-f97737bbbca7 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:03.727786 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-e1a8d1a5-0f98-4cc5-95a1-f97737bbbca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:03.733409 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8d1a5-0f98-4cc5-95a1-f97737bbbca7 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:03.795537 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792f375-7992-4bae-8cb7-ef07efa1fb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:03.796183 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792f375-7992-4bae-8cb7-ef07efa1fb03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:03.796459 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792f375-7992-4bae-8cb7-ef07efa1fb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:03.796728 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792f375-7992-4bae-8cb7-ef07efa1fb03 admin admin] PATH_INFO: `/v3/users/4726a0fcd9134abda9cf1f73958e2235` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:03.797878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5792f375-7992-4bae-8cb7-ef07efa1fb03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4726a0fcd9134abda9cf1f73958e2235)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:03.799901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5792f375-7992-4bae-8cb7-ef07efa1fb03 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:03.986474 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5792f375-7992-4bae-8cb7-ef07efa1fb03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4726a0fcd9134abda9cf1f73958e2235'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:04.099237 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5792f375-7992-4bae-8cb7-ef07efa1fb03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4726a0fcd9134abda9cf1f73958e2235'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:04.099963 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5792f375-7992-4bae-8cb7-ef07efa1fb03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4726a0fcd9134abda9cf1f73958e2235'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:04.110080 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5792f375-7992-4bae-8cb7-ef07efa1fb03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4726a0fcd9134abda9cf1f73958e2235'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:04.125918 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-5792f375-7992-4bae-8cb7-ef07efa1fb03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4726a0fcd9134abda9cf1f73958e2235'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:04.134358 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1411/2820] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:03:03 2022] DELETE /identity/v3/users/4726a0fcd9134abda9cf1f73958e2235 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:04.618259 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc22444-12d6-4078-95d4-317b3d420201 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:04.620127 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-fcc22444-12d6-4078-95d4-317b3d420201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:04.634036 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc22444-12d6-4078-95d4-317b3d420201 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:04.698254 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107147e8-6ec6-4d60-8d34-3d59e69fd4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:04.698915 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107147e8-6ec6-4d60-8d34-3d59e69fd4f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:04.699188 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107147e8-6ec6-4d60-8d34-3d59e69fd4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:04.699450 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107147e8-6ec6-4d60-8d34-3d59e69fd4f8 admin admin] PATH_INFO: `/v3/projects/f1e12c012fd6420d8e049ba6f5de2aff` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:04.700618 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107147e8-6ec6-4d60-8d34-3d59e69fd4f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1e12c012fd6420d8e049ba6f5de2aff)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:04.702552 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107147e8-6ec6-4d60-8d34-3d59e69fd4f8 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:04.828248 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-107147e8-6ec6-4d60-8d34-3d59e69fd4f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1e12c012fd6420d8e049ba6f5de2aff', 'request_id': 'req-107147e8-6ec6-4d60-8d34-3d59e69fd4f8'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:04.834080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-107147e8-6ec6-4d60-8d34-3d59e69fd4f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1e12c012fd6420d8e049ba6f5de2aff', 'request_id': 'req-107147e8-6ec6-4d60-8d34-3d59e69fd4f8'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:04.841685 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-107147e8-6ec6-4d60-8d34-3d59e69fd4f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1e12c012fd6420d8e049ba6f5de2aff', 'request_id': 'req-107147e8-6ec6-4d60-8d34-3d59e69fd4f8'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:04.905214 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-107147e8-6ec6-4d60-8d34-3d59e69fd4f8 admin admin] The token cache is being invalidate because project f1e12c012fd6420d8e049ba6f5de2aff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:03:04.905448 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-107147e8-6ec6-4d60-8d34-3d59e69fd4f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:04.906044 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-107147e8-6ec6-4d60-8d34-3d59e69fd4f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:04.907584 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1410/2821] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:03:04 2022] DELETE /identity/v3/projects/f1e12c012fd6420d8e049ba6f5de2aff => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:04.913729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4256f39-db7d-40cb-92c3-5c357e1ddbd6 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:04.914862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4256f39-db7d-40cb-92c3-5c357e1ddbd6 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:04.914862 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4256f39-db7d-40cb-92c3-5c357e1ddbd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:04.937982 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-e4256f39-db7d-40cb-92c3-5c357e1ddbd6 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:04.937982 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-e4256f39-db7d-40cb-92c3-5c357e1ddbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:05.006706 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1412/2822] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:03:05.015919 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53661a0-1212-4d31-bf7b-a89b1f57a28b None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:05.016202 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53661a0-1212-4d31-bf7b-a89b1f57a28b None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:05.016480 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53661a0-1212-4d31-bf7b-a89b1f57a28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:05.036740 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-e53661a0-1212-4d31-bf7b-a89b1f57a28b None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:05.037952 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-e53661a0-1212-4d31-bf7b-a89b1f57a28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:05.046385 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1411/2823] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:05.051893 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4256f39-db7d-40cb-92c3-5c357e1ddbd6 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:05.060566 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:05.070218 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4256f39-db7d-40cb-92c3-5c357e1ddbd6 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:05.078127 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754d84d-f9c8-4dd4-be35-a41daa660721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:05.079288 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754d84d-f9c8-4dd4-be35-a41daa660721 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:05.079864 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754d84d-f9c8-4dd4-be35-a41daa660721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:05.080387 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754d84d-f9c8-4dd4-be35-a41daa660721 admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:05.081500 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754d84d-f9c8-4dd4-be35-a41daa660721 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:03:05.082043 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754d84d-f9c8-4dd4-be35-a41daa660721 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:05.085193 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754d84d-f9c8-4dd4-be35-a41daa660721 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:05.101942 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1413/2824] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 17:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:03:05.108616 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53661a0-1212-4d31-bf7b-a89b1f57a28b None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:05.119815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53661a0-1212-4d31-bf7b-a89b1f57a28b None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:05.124826 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc368ca8-929d-41c1-879a-be52422e7d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:05.125478 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc368ca8-929d-41c1-879a-be52422e7d1a admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:05.125793 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc368ca8-929d-41c1-879a-be52422e7d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:05.126059 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc368ca8-929d-41c1-879a-be52422e7d1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:05.126794 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc368ca8-929d-41c1-879a-be52422e7d1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:05.128775 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc368ca8-929d-41c1-879a-be52422e7d1a admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:05.221501 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1412/2825] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:03:05 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:03:05.227623 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754d84d-f9c8-4dd4-be35-a41daa660721 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:05.238868 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754d84d-f9c8-4dd4-be35-a41daa660721 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:05.328421 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2626d-054b-49be-9172-1c09e418dbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:05.329051 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2626d-054b-49be-9172-1c09e418dbbd admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:05.329332 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2626d-054b-49be-9172-1c09e418dbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:05.329600 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2626d-054b-49be-9172-1c09e418dbbd admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:05.330235 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2626d-054b-49be-9172-1c09e418dbbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:05.332230 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2626d-054b-49be-9172-1c09e418dbbd admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:05.464137 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1414/2826] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:03:05 2022] POST /identity/v3/users => generated 290 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:03:05.472186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc368ca8-929d-41c1-879a-be52422e7d1a admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:05.485573 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc368ca8-929d-41c1-879a-be52422e7d1a admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:05.490605 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd18de-305e-4dee-888a-a3262cda0deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:05.491234 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd18de-305e-4dee-888a-a3262cda0deb admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:05.491511 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd18de-305e-4dee-888a-a3262cda0deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:05.491779 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd18de-305e-4dee-888a-a3262cda0deb admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:05.492432 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd18de-305e-4dee-888a-a3262cda0deb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:05.494817 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd18de-305e-4dee-888a-a3262cda0deb admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:05.505735 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1413/2827] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:03:05.510734 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2626d-054b-49be-9172-1c09e418dbbd admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:05.524857 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2626d-054b-49be-9172-1c09e418dbbd admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:05.530188 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d31f9-3de1-4107-aae3-eaf9593be14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:05.530847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d31f9-3de1-4107-aae3-eaf9593be14c admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:05.531123 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d31f9-3de1-4107-aae3-eaf9593be14c admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:05.531390 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d31f9-3de1-4107-aae3-eaf9593be14c admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:05.532094 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32d31f9-3de1-4107-aae3-eaf9593be14c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:05.533697 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32d31f9-3de1-4107-aae3-eaf9593be14c admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:05.546444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1415/2828] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:03:05.551104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd18de-305e-4dee-888a-a3262cda0deb admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:05.563792 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd18de-305e-4dee-888a-a3262cda0deb admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:05.568501 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a81bb7-c4bc-4d5c-8b71-b04db270d1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:05.569186 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a81bb7-c4bc-4d5c-8b71-b04db270d1c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:05.569465 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a81bb7-c4bc-4d5c-8b71-b04db270d1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:05.569734 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a81bb7-c4bc-4d5c-8b71-b04db270d1c4 admin admin] PATH_INFO: `/v3/projects/d8872a2d46344dce981c6dffe1504200/users/f99978f3c89147448a512b8a7e7fdeb6/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:05.580337 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:05.588864 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a81bb7-c4bc-4d5c-8b71-b04db270d1c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8872a2d46344dce981c6dffe1504200, user_id=f99978f3c89147448a512b8a7e7fdeb6, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:05.603670 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a81bb7-c4bc-4d5c-8b71-b04db270d1c4 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:05.715024 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1414/2829] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:03:05 2022] PUT /identity/v3/projects/d8872a2d46344dce981c6dffe1504200/users/f99978f3c89147448a512b8a7e7fdeb6/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:05.788469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6284c7-7e6d-440b-ac1a-77ed4d03d925 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:05.788698 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6284c7-7e6d-440b-ac1a-77ed4d03d925 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:05.788922 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6284c7-7e6d-440b-ac1a-77ed4d03d925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:05.820983 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-3f6284c7-7e6d-440b-ac1a-77ed4d03d925 None None] MFA Rules not processed for user `f99978f3c89147448a512b8a7e7fdeb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:05.823124 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-3f6284c7-7e6d-440b-ac1a-77ed4d03d925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:05.889447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1416/2830] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:03:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:05.894177 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a81bb7-c4bc-4d5c-8b71-b04db270d1c4 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:05.963238 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a81bb7-c4bc-4d5c-8b71-b04db270d1c4 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:05.967927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf9319-67de-4c42-be50-8ecbd48d9662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:05.968532 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf9319-67de-4c42-be50-8ecbd48d9662 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:05.968807 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf9319-67de-4c42-be50-8ecbd48d9662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:05.969070 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf9319-67de-4c42-be50-8ecbd48d9662 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:05.969691 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf9319-67de-4c42-be50-8ecbd48d9662 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:05.971643 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf9319-67de-4c42-be50-8ecbd48d9662 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:06.110595 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1415/2831] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:03:05 2022] POST /identity/v3/projects => generated 379 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:03:06.115199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6284c7-7e6d-440b-ac1a-77ed4d03d925 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:06.127440 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6284c7-7e6d-440b-ac1a-77ed4d03d925 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:06.191456 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:06.198495 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f0209-d96d-4a38-8dff-4fd50638fcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:06.199161 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f0209-d96d-4a38-8dff-4fd50638fcd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:06.199442 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f0209-d96d-4a38-8dff-4fd50638fcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:06.199711 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f0209-d96d-4a38-8dff-4fd50638fcd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:06.200344 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f0209-d96d-4a38-8dff-4fd50638fcd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:06.202265 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f0209-d96d-4a38-8dff-4fd50638fcd4 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:06.291447 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1417/2832] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:03:06 2022] POST /identity/v3/users => generated 296 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:03:06.296056 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf9319-67de-4c42-be50-8ecbd48d9662 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:06.306347 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf9319-67de-4c42-be50-8ecbd48d9662 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:06.312016 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e7368-9087-490a-935b-63085e366940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:06.312621 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e7368-9087-490a-935b-63085e366940 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:06.312896 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e7368-9087-490a-935b-63085e366940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:06.313162 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e7368-9087-490a-935b-63085e366940 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:06.313788 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e7368-9087-490a-935b-63085e366940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:06.315430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e7368-9087-490a-935b-63085e366940 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:06.326398 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1416/2833] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:03:06.330853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f0209-d96d-4a38-8dff-4fd50638fcd4 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:06.341508 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f0209-d96d-4a38-8dff-4fd50638fcd4 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:06.346227 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9a313-67e0-446f-9f0b-c78cdce036be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:06.346961 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9a313-67e0-446f-9f0b-c78cdce036be admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:06.347239 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9a313-67e0-446f-9f0b-c78cdce036be admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:06.347512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9a313-67e0-446f-9f0b-c78cdce036be admin admin] PATH_INFO: `/v3/projects/506b95099e22462886f2d7c872398ee2/users/179b44352ddd4c0f9c5b5a5eb3bd5fef/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:06.363111 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9a313-67e0-446f-9f0b-c78cdce036be admin admin] RBAC: Authorizing `identity:create_grant(project_id=506b95099e22462886f2d7c872398ee2, user_id=179b44352ddd4c0f9c5b5a5eb3bd5fef, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:06.365770 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9a313-67e0-446f-9f0b-c78cdce036be admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:06.502826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1418/2834] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:03:06 2022] PUT /identity/v3/projects/506b95099e22462886f2d7c872398ee2/users/179b44352ddd4c0f9c5b5a5eb3bd5fef/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:06.507033 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e7368-9087-490a-935b-63085e366940 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:06.517429 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e7368-9087-490a-935b-63085e366940 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:06.580694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ef164-00e4-4c55-9da3-dcca2945f38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:06.581325 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ef164-00e4-4c55-9da3-dcca2945f38e admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:06.581602 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ef164-00e4-4c55-9da3-dcca2945f38e admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:06.581875 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ef164-00e4-4c55-9da3-dcca2945f38e admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:06.582526 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ef164-00e4-4c55-9da3-dcca2945f38e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:06.584218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ef164-00e4-4c55-9da3-dcca2945f38e admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:06.597311 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1417/2835] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:03:06.601702 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9a313-67e0-446f-9f0b-c78cdce036be admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:06.614411 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9a313-67e0-446f-9f0b-c78cdce036be admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:06.619294 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000491aa-c450-4d23-8e5e-e4aa3c703870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:06.619957 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000491aa-c450-4d23-8e5e-e4aa3c703870 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:06.620233 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000491aa-c450-4d23-8e5e-e4aa3c703870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:06.620498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000491aa-c450-4d23-8e5e-e4aa3c703870 admin admin] PATH_INFO: `/v3/projects/506b95099e22462886f2d7c872398ee2/users/179b44352ddd4c0f9c5b5a5eb3bd5fef/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:06.623229 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000491aa-c450-4d23-8e5e-e4aa3c703870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=506b95099e22462886f2d7c872398ee2, user_id=179b44352ddd4c0f9c5b5a5eb3bd5fef, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:06.625906 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000491aa-c450-4d23-8e5e-e4aa3c703870 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:06.637443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1419/2836] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:03:06 2022] PUT /identity/v3/projects/506b95099e22462886f2d7c872398ee2/users/179b44352ddd4c0f9c5b5a5eb3bd5fef/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:06.648097 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360de00-b0e1-44ed-ad0d-7040c73bcea7 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:06.648476 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360de00-b0e1-44ed-ad0d-7040c73bcea7 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:06.648780 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360de00-b0e1-44ed-ad0d-7040c73bcea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:06.680826 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-2360de00-b0e1-44ed-ad0d-7040c73bcea7 None None] MFA Rules not processed for user `179b44352ddd4c0f9c5b5a5eb3bd5fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:06.682057 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-2360de00-b0e1-44ed-ad0d-7040c73bcea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:06.728691 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:06.755414 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1418/2837] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:03:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:06.768737 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000491aa-c450-4d23-8e5e-e4aa3c703870 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:06.770531 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-000491aa-c450-4d23-8e5e-e4aa3c703870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:06.836567 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000491aa-c450-4d23-8e5e-e4aa3c703870 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:06.840856 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb100d-08dd-467c-ac68-b296d54e8393 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:06.841625 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb100d-08dd-467c-ac68-b296d54e8393 service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:06.841907 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb100d-08dd-467c-ac68-b296d54e8393 service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:06.842185 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb100d-08dd-467c-ac68-b296d54e8393 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:06.854024 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb100d-08dd-467c-ac68-b296d54e8393 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:06.856292 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb100d-08dd-467c-ac68-b296d54e8393 service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:06.871270 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1420/2838] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:03:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:03:07.451753 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360de00-b0e1-44ed-ad0d-7040c73bcea7 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:07.533675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360de00-b0e1-44ed-ad0d-7040c73bcea7 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:07.538482 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241cbe6-d5bc-4a71-a9e4-16e79cda31e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:07.540052 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241cbe6-d5bc-4a71-a9e4-16e79cda31e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:07.540052 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241cbe6-d5bc-4a71-a9e4-16e79cda31e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:07.540052 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241cbe6-d5bc-4a71-a9e4-16e79cda31e6 admin admin] PATH_INFO: `/v3/users/f99978f3c89147448a512b8a7e7fdeb6` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:07.540808 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241cbe6-d5bc-4a71-a9e4-16e79cda31e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f99978f3c89147448a512b8a7e7fdeb6)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:07.543442 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241cbe6-d5bc-4a71-a9e4-16e79cda31e6 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:07.745193 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-8241cbe6-d5bc-4a71-a9e4-16e79cda31e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f99978f3c89147448a512b8a7e7fdeb6'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:07.836418 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-8241cbe6-d5bc-4a71-a9e4-16e79cda31e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f99978f3c89147448a512b8a7e7fdeb6'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:07.837140 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-8241cbe6-d5bc-4a71-a9e4-16e79cda31e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f99978f3c89147448a512b8a7e7fdeb6'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:07.847714 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-8241cbe6-d5bc-4a71-a9e4-16e79cda31e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f99978f3c89147448a512b8a7e7fdeb6'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:07.848440 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-8241cbe6-d5bc-4a71-a9e4-16e79cda31e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f99978f3c89147448a512b8a7e7fdeb6'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:07.870324 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1419/2839] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:03:07 2022] DELETE /identity/v3/users/f99978f3c89147448a512b8a7e7fdeb6 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:07.883178 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a30ad-f91d-46e3-9a21-4d9ab2bad5e6 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:07.883178 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a30ad-f91d-46e3-9a21-4d9ab2bad5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:07.883178 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a30ad-f91d-46e3-9a21-4d9ab2bad5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:07.901765 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-c27a30ad-f91d-46e3-9a21-4d9ab2bad5e6 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:07.904167 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-c27a30ad-f91d-46e3-9a21-4d9ab2bad5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:07.969852 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1421/2840] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:07.980128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241cbe6-d5bc-4a71-a9e4-16e79cda31e6 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:07.981630 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-8241cbe6-d5bc-4a71-a9e4-16e79cda31e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:08.047073 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241cbe6-d5bc-4a71-a9e4-16e79cda31e6 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:08.050744 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ea4a5-540f-4ff2-99a1-2711703ef44e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:08.051344 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ea4a5-540f-4ff2-99a1-2711703ef44e service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:08.051621 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ea4a5-540f-4ff2-99a1-2711703ef44e service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:08.051885 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ea4a5-540f-4ff2-99a1-2711703ef44e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:08.062426 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ea4a5-540f-4ff2-99a1-2711703ef44e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:08.063894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ea4a5-540f-4ff2-99a1-2711703ef44e service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:08.074302 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1420/2841] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:03:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:03:08.581063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a30ad-f91d-46e3-9a21-4d9ab2bad5e6 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:08.582669 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-c27a30ad-f91d-46e3-9a21-4d9ab2bad5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:08.597117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a30ad-f91d-46e3-9a21-4d9ab2bad5e6 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:08.604617 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1d21b-d137-42f1-80d3-5321a913490d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:08.606812 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1d21b-d137-42f1-80d3-5321a913490d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:08.607051 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1d21b-d137-42f1-80d3-5321a913490d admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:08.607354 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1d21b-d137-42f1-80d3-5321a913490d admin admin] PATH_INFO: `/v3/projects/d8872a2d46344dce981c6dffe1504200` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:08.608555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1d21b-d137-42f1-80d3-5321a913490d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8872a2d46344dce981c6dffe1504200)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:08.610695 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1d21b-d137-42f1-80d3-5321a913490d admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:08.757916 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:08.759174 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-d7c1d21b-d137-42f1-80d3-5321a913490d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8872a2d46344dce981c6dffe1504200', 'request_id': 'req-d7c1d21b-d137-42f1-80d3-5321a913490d'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:08.880126 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-d7c1d21b-d137-42f1-80d3-5321a913490d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8872a2d46344dce981c6dffe1504200', 'request_id': 'req-d7c1d21b-d137-42f1-80d3-5321a913490d'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:08.887267 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-d7c1d21b-d137-42f1-80d3-5321a913490d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8872a2d46344dce981c6dffe1504200', 'request_id': 'req-d7c1d21b-d137-42f1-80d3-5321a913490d'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:08.898201 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-d7c1d21b-d137-42f1-80d3-5321a913490d admin admin] The token cache is being invalidate because project d8872a2d46344dce981c6dffe1504200 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:03:08.898489 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-d7c1d21b-d137-42f1-80d3-5321a913490d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:08.899178 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-d7c1d21b-d137-42f1-80d3-5321a913490d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:08.900672 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1422/2842] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:03:08 2022] DELETE /identity/v3/projects/d8872a2d46344dce981c6dffe1504200 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:08.905438 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ea4a5-540f-4ff2-99a1-2711703ef44e service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:08.911675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:08.915539 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ea4a5-540f-4ff2-99a1-2711703ef44e service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:08.982700 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f65435-af11-4335-a5e5-981b7a5f7c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:08.983398 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f65435-af11-4335-a5e5-981b7a5f7c17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:08.983836 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f65435-af11-4335-a5e5-981b7a5f7c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:08.984215 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f65435-af11-4335-a5e5-981b7a5f7c17 admin admin] PATH_INFO: `/v3/users/179b44352ddd4c0f9c5b5a5eb3bd5fef` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:08.986420 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f65435-af11-4335-a5e5-981b7a5f7c17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=179b44352ddd4c0f9c5b5a5eb3bd5fef)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:08.990183 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f65435-af11-4335-a5e5-981b7a5f7c17 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:09.259552 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-86f65435-af11-4335-a5e5-981b7a5f7c17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '179b44352ddd4c0f9c5b5a5eb3bd5fef'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:09.348023 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-86f65435-af11-4335-a5e5-981b7a5f7c17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '179b44352ddd4c0f9c5b5a5eb3bd5fef'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:09.348767 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-86f65435-af11-4335-a5e5-981b7a5f7c17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '179b44352ddd4c0f9c5b5a5eb3bd5fef'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:09.356811 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-86f65435-af11-4335-a5e5-981b7a5f7c17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '179b44352ddd4c0f9c5b5a5eb3bd5fef'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:09.357521 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-86f65435-af11-4335-a5e5-981b7a5f7c17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '179b44352ddd4c0f9c5b5a5eb3bd5fef'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:09.375626 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1421/2843] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:03:08 2022] DELETE /identity/v3/users/179b44352ddd4c0f9c5b5a5eb3bd5fef => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:03:10.042446 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1d21b-d137-42f1-80d3-5321a913490d admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:10.045251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-d7c1d21b-d137-42f1-80d3-5321a913490d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:10.066010 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1d21b-d137-42f1-80d3-5321a913490d admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:10.132978 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329acd0-ec77-4a2e-a361-6b328e18ffe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:10.133625 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329acd0-ec77-4a2e-a361-6b328e18ffe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:10.133901 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329acd0-ec77-4a2e-a361-6b328e18ffe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:10.134166 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329acd0-ec77-4a2e-a361-6b328e18ffe9 admin admin] PATH_INFO: `/v3/projects/506b95099e22462886f2d7c872398ee2` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:10.135369 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0329acd0-ec77-4a2e-a361-6b328e18ffe9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=506b95099e22462886f2d7c872398ee2)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:10.137321 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0329acd0-ec77-4a2e-a361-6b328e18ffe9 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:10.249884 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0329acd0-ec77-4a2e-a361-6b328e18ffe9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '506b95099e22462886f2d7c872398ee2', 'request_id': 'req-0329acd0-ec77-4a2e-a361-6b328e18ffe9'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:10.342974 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0329acd0-ec77-4a2e-a361-6b328e18ffe9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '506b95099e22462886f2d7c872398ee2', 'request_id': 'req-0329acd0-ec77-4a2e-a361-6b328e18ffe9'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:10.349831 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0329acd0-ec77-4a2e-a361-6b328e18ffe9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '506b95099e22462886f2d7c872398ee2', 'request_id': 'req-0329acd0-ec77-4a2e-a361-6b328e18ffe9'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:10.361072 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0329acd0-ec77-4a2e-a361-6b328e18ffe9 admin admin] The token cache is being invalidate because project 506b95099e22462886f2d7c872398ee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:03:10.361402 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0329acd0-ec77-4a2e-a361-6b328e18ffe9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:10.361999 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-0329acd0-ec77-4a2e-a361-6b328e18ffe9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:10.363522 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1423/2844] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:03:10 2022] DELETE /identity/v3/projects/506b95099e22462886f2d7c872398ee2 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:10.369509 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79076a-5abd-4c18-be3b-53a437bd776f None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:10.369992 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79076a-5abd-4c18-be3b-53a437bd776f None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:10.370112 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79076a-5abd-4c18-be3b-53a437bd776f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:10.389804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-fa79076a-5abd-4c18-be3b-53a437bd776f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:10.390973 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-fa79076a-5abd-4c18-be3b-53a437bd776f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:10.450724 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1422/2845] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:03:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:10.465247 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd758a-ed61-45fe-99d4-122dd54b7e5d None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:10.465528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd758a-ed61-45fe-99d4-122dd54b7e5d None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:10.466406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd758a-ed61-45fe-99d4-122dd54b7e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:10.489576 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-edfd758a-ed61-45fe-99d4-122dd54b7e5d None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:10.490920 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-edfd758a-ed61-45fe-99d4-122dd54b7e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:10.499344 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1424/2846] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:03:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:10.503865 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79076a-5abd-4c18-be3b-53a437bd776f None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:10.514404 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79076a-5abd-4c18-be3b-53a437bd776f None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:10.519030 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e443de-aaae-4060-9e78-d91630e490f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:10.519632 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e443de-aaae-4060-9e78-d91630e490f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:10.519906 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e443de-aaae-4060-9e78-d91630e490f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:10.520171 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e443de-aaae-4060-9e78-d91630e490f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:10.520815 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e443de-aaae-4060-9e78-d91630e490f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81998) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:03:10.521106 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e443de-aaae-4060-9e78-d91630e490f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:10.522726 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e443de-aaae-4060-9e78-d91630e490f7 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:10.535962 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1423/2847] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 17:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:03:10.540371 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd758a-ed61-45fe-99d4-122dd54b7e5d None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:10.551199 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd758a-ed61-45fe-99d4-122dd54b7e5d None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:10.556132 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb8a1b-02e5-4362-81db-e3e618bc90c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:10.556875 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb8a1b-02e5-4362-81db-e3e618bc90c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:10.557240 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb8a1b-02e5-4362-81db-e3e618bc90c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:10.557605 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb8a1b-02e5-4362-81db-e3e618bc90c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:10.558327 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb8a1b-02e5-4362-81db-e3e618bc90c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:10.561366 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb8a1b-02e5-4362-81db-e3e618bc90c3 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:10.666725 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1425/2848] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:03:10 2022] POST /identity/v3/projects => generated 363 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:03:10.671458 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e443de-aaae-4060-9e78-d91630e490f7 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:10.682174 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e443de-aaae-4060-9e78-d91630e490f7 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:10.691706 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:10.757198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967761d-5238-4cda-b2d5-08bb02a757da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:10.757939 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967761d-5238-4cda-b2d5-08bb02a757da admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:10.758074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967761d-5238-4cda-b2d5-08bb02a757da admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:10.758336 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967761d-5238-4cda-b2d5-08bb02a757da admin admin] PATH_INFO: `/v3/users` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:10.759020 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967761d-5238-4cda-b2d5-08bb02a757da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:10.760961 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967761d-5238-4cda-b2d5-08bb02a757da admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:10.886079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1424/2849] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:03:10 2022] POST /identity/v3/users => generated 282 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:03:10.891220 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb8a1b-02e5-4362-81db-e3e618bc90c3 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:10.901900 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb8a1b-02e5-4362-81db-e3e618bc90c3 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:10.907264 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c47ca-40ef-4a17-a0de-8582ebd4e2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:10.907882 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c47ca-40ef-4a17-a0de-8582ebd4e2b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:10.908150 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c47ca-40ef-4a17-a0de-8582ebd4e2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:10.908421 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c47ca-40ef-4a17-a0de-8582ebd4e2b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:10.909057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c47ca-40ef-4a17-a0de-8582ebd4e2b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:10.910671 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c47ca-40ef-4a17-a0de-8582ebd4e2b3 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:10.921514 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1426/2850] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:03:10.925817 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967761d-5238-4cda-b2d5-08bb02a757da admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:10.939052 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967761d-5238-4cda-b2d5-08bb02a757da admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:10.943835 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3ee07-a30e-4687-9bf2-d03ad31e0dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:10.944443 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3ee07-a30e-4687-9bf2-d03ad31e0dee admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:10.944717 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3ee07-a30e-4687-9bf2-d03ad31e0dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:10.944988 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3ee07-a30e-4687-9bf2-d03ad31e0dee admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:10.945625 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c3ee07-a30e-4687-9bf2-d03ad31e0dee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:10.947255 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c3ee07-a30e-4687-9bf2-d03ad31e0dee admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:10.960215 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1425/2851] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:03:10.964743 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c47ca-40ef-4a17-a0de-8582ebd4e2b3 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:10.975500 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c47ca-40ef-4a17-a0de-8582ebd4e2b3 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:10.980184 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384e9cc-e2bd-44d6-8fe7-31eb609e1e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:10.980853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384e9cc-e2bd-44d6-8fe7-31eb609e1e0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:10.981121 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384e9cc-e2bd-44d6-8fe7-31eb609e1e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:10.981655 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384e9cc-e2bd-44d6-8fe7-31eb609e1e0c admin admin] PATH_INFO: `/v3/projects/4ec5588e52df4d609e39fec525138f01/users/c94ac68dd4b241e9a7119e11be6a28f9/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:10.999189 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f384e9cc-e2bd-44d6-8fe7-31eb609e1e0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ec5588e52df4d609e39fec525138f01, user_id=c94ac68dd4b241e9a7119e11be6a28f9, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:11.001849 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f384e9cc-e2bd-44d6-8fe7-31eb609e1e0c admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:11.116847 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1427/2852] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:03:10 2022] PUT /identity/v3/projects/4ec5588e52df4d609e39fec525138f01/users/c94ac68dd4b241e9a7119e11be6a28f9/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:11.131275 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745c0ce-d663-4ef4-a05e-d95f059ad578 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:11.131861 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745c0ce-d663-4ef4-a05e-d95f059ad578 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:11.132198 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745c0ce-d663-4ef4-a05e-d95f059ad578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:11.177292 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-d745c0ce-d663-4ef4-a05e-d95f059ad578 None None] MFA Rules not processed for user `c94ac68dd4b241e9a7119e11be6a28f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:11.178529 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d745c0ce-d663-4ef4-a05e-d95f059ad578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:11.256894 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1426/2853] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:03:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:11.270291 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384e9cc-e2bd-44d6-8fe7-31eb609e1e0c admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:11.272014 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-f384e9cc-e2bd-44d6-8fe7-31eb609e1e0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:11.278176 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:11.336217 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384e9cc-e2bd-44d6-8fe7-31eb609e1e0c admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:11.340223 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370769ac-471b-4505-bc42-926f67544647 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bbf0678f34e46e8b56aaeabce955180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:11.340897 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370769ac-471b-4505-bc42-926f67544647 service cinder] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:11.341394 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370769ac-471b-4505-bc42-926f67544647 service cinder] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:11.341826 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370769ac-471b-4505-bc42-926f67544647 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:11.352176 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370769ac-471b-4505-bc42-926f67544647 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:11.353638 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370769ac-471b-4505-bc42-926f67544647 service cinder] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:11.364493 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1428/2854] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:03:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:03:21.253612 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=61,keystone:INSERT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:21.384726 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=25 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:26.864746 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745c0ce-d663-4ef4-a05e-d95f059ad578 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:26.868281 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d745c0ce-d663-4ef4-a05e-d95f059ad578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:27.009104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745c0ce-d663-4ef4-a05e-d95f059ad578 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:27.015184 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc67507-e2e6-4ef3-af80-d1df83c4ebc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:27.015808 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc67507-e2e6-4ef3-af80-d1df83c4ebc5 service nova] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:27.016199 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc67507-e2e6-4ef3-af80-d1df83c4ebc5 service nova] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:27.016371 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc67507-e2e6-4ef3-af80-d1df83c4ebc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:27.018283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-3bc67507-e2e6-4ef3-af80-d1df83c4ebc5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:27.032500 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc67507-e2e6-4ef3-af80-d1df83c4ebc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:27.034192 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc67507-e2e6-4ef3-af80-d1df83c4ebc5 service nova] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:27.047689 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1427/2855] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:03:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:03:27.128039 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370769ac-471b-4505-bc42-926f67544647 service cinder] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:27.129666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-370769ac-471b-4505-bc42-926f67544647 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:27.198315 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370769ac-471b-4505-bc42-926f67544647 service cinder] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:27.203597 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9509b2-6d19-468d-9432-a969f1ad540f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:27.204240 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9509b2-6d19-468d-9432-a969f1ad540f service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:27.204512 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9509b2-6d19-468d-9432-a969f1ad540f service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:27.204780 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9509b2-6d19-468d-9432-a969f1ad540f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:27.206706 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-1c9509b2-6d19-468d-9432-a969f1ad540f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:27.218444 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9509b2-6d19-468d-9432-a969f1ad540f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:27.219924 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9509b2-6d19-468d-9432-a969f1ad540f service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:27.229631 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1429/2856] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:03:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:03:28.611869 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc67507-e2e6-4ef3-af80-d1df83c4ebc5 service nova] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:28.613354 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-3bc67507-e2e6-4ef3-af80-d1df83c4ebc5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:28.628291 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc67507-e2e6-4ef3-af80-d1df83c4ebc5 service nova] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:28.633687 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb4a18-5308-4f4c-9449-e02ec91c1797 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:28.634284 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb4a18-5308-4f4c-9449-e02ec91c1797 service nova] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:28.634559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb4a18-5308-4f4c-9449-e02ec91c1797 service nova] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:28.634850 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb4a18-5308-4f4c-9449-e02ec91c1797 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:28.636564 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-b9bb4a18-5308-4f4c-9449-e02ec91c1797 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:28.706544 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb4a18-5308-4f4c-9449-e02ec91c1797 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:28.708202 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb4a18-5308-4f4c-9449-e02ec91c1797 service nova] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:28.718064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1428/2857] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:03:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:03:29.636051 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9509b2-6d19-468d-9432-a969f1ad540f service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:29.650295 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9509b2-6d19-468d-9432-a969f1ad540f service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:29.657031 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84fbcd-dcf3-4306-bec5-0d04b5d1bec5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:29.657640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84fbcd-dcf3-4306-bec5-0d04b5d1bec5 service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:29.657920 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84fbcd-dcf3-4306-bec5-0d04b5d1bec5 service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:29.658191 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84fbcd-dcf3-4306-bec5-0d04b5d1bec5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:29.660022 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ac84fbcd-dcf3-4306-bec5-0d04b5d1bec5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:29.674100 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84fbcd-dcf3-4306-bec5-0d04b5d1bec5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:29.675741 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84fbcd-dcf3-4306-bec5-0d04b5d1bec5 service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:29.687626 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1430/2858] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:03:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:03:32.645884 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb4a18-5308-4f4c-9449-e02ec91c1797 service nova] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:32.657783 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb4a18-5308-4f4c-9449-e02ec91c1797 service nova] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:32.661619 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f39f9-d541-4ff9-9bf2-db03bd621fcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:32.662303 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f39f9-d541-4ff9-9bf2-db03bd621fcb service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:32.662698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f39f9-d541-4ff9-9bf2-db03bd621fcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:32.663057 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f39f9-d541-4ff9-9bf2-db03bd621fcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:32.665167 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d65f39f9-d541-4ff9-9bf2-db03bd621fcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:32.743355 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f39f9-d541-4ff9-9bf2-db03bd621fcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:32.744804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f39f9-d541-4ff9-9bf2-db03bd621fcb service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:32.755432 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1429/2859] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:03:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:03:39.682071 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=32 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:40.604978 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84fbcd-dcf3-4306-bec5-0d04b5d1bec5 service nova] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:40.607672 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ac84fbcd-dcf3-4306-bec5-0d04b5d1bec5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:40.621096 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84fbcd-dcf3-4306-bec5-0d04b5d1bec5 service nova] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:40.626251 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd4ff8-2a91-4903-87a3-2be4bc711de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:40.627022 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd4ff8-2a91-4903-87a3-2be4bc711de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:40.627297 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd4ff8-2a91-4903-87a3-2be4bc711de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:40.627589 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd4ff8-2a91-4903-87a3-2be4bc711de6 admin admin] PATH_INFO: `/v3/users/803b15aebfb044c3893831f84f1aa1d7` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:40.628713 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd4ff8-2a91-4903-87a3-2be4bc711de6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=803b15aebfb044c3893831f84f1aa1d7)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:40.630634 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd4ff8-2a91-4903-87a3-2be4bc711de6 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:40.817322 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-a8cd4ff8-2a91-4903-87a3-2be4bc711de6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '803b15aebfb044c3893831f84f1aa1d7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:40.863873 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-a8cd4ff8-2a91-4903-87a3-2be4bc711de6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '803b15aebfb044c3893831f84f1aa1d7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:40.864484 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-a8cd4ff8-2a91-4903-87a3-2be4bc711de6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '803b15aebfb044c3893831f84f1aa1d7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:40.871658 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-a8cd4ff8-2a91-4903-87a3-2be4bc711de6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '803b15aebfb044c3893831f84f1aa1d7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:40.884423 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-a8cd4ff8-2a91-4903-87a3-2be4bc711de6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '803b15aebfb044c3893831f84f1aa1d7'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:40.890883 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1431/2860] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:03:40 2022] DELETE /identity/v3/users/803b15aebfb044c3893831f84f1aa1d7 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:41.163398 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f39f9-d541-4ff9-9bf2-db03bd621fcb service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:41.187660 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f39f9-d541-4ff9-9bf2-db03bd621fcb service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:41.197130 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=100 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:41.246043 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553242e-ff6e-4df2-b7af-92364ecb2b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:41.246698 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553242e-ff6e-4df2-b7af-92364ecb2b72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:41.246975 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553242e-ff6e-4df2-b7af-92364ecb2b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:41.247242 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553242e-ff6e-4df2-b7af-92364ecb2b72 admin admin] PATH_INFO: `/v3/projects/73ca0765d1fb44cda1c4c2562a695ac6` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:41.248346 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553242e-ff6e-4df2-b7af-92364ecb2b72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73ca0765d1fb44cda1c4c2562a695ac6)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:41.251263 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553242e-ff6e-4df2-b7af-92364ecb2b72 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:41.356064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-9553242e-ff6e-4df2-b7af-92364ecb2b72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73ca0765d1fb44cda1c4c2562a695ac6', 'request_id': 'req-9553242e-ff6e-4df2-b7af-92364ecb2b72'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:41.362433 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-9553242e-ff6e-4df2-b7af-92364ecb2b72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73ca0765d1fb44cda1c4c2562a695ac6', 'request_id': 'req-9553242e-ff6e-4df2-b7af-92364ecb2b72'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:41.369083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-9553242e-ff6e-4df2-b7af-92364ecb2b72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73ca0765d1fb44cda1c4c2562a695ac6', 'request_id': 'req-9553242e-ff6e-4df2-b7af-92364ecb2b72'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:41.475264 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-9553242e-ff6e-4df2-b7af-92364ecb2b72 admin admin] The token cache is being invalidate because project 73ca0765d1fb44cda1c4c2562a695ac6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:03:41.475609 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-9553242e-ff6e-4df2-b7af-92364ecb2b72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:41.476236 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-9553242e-ff6e-4df2-b7af-92364ecb2b72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:41.477748 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1430/2861] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:03:41 2022] DELETE /identity/v3/projects/73ca0765d1fb44cda1c4c2562a695ac6 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:41.481875 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd4ff8-2a91-4903-87a3-2be4bc711de6 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:41.483422 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-a8cd4ff8-2a91-4903-87a3-2be4bc711de6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:41.497681 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd4ff8-2a91-4903-87a3-2be4bc711de6 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:41.555517 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9300e4-8f81-4524-a859-fd701f6916d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:41.556146 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9300e4-8f81-4524-a859-fd701f6916d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:41.556424 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9300e4-8f81-4524-a859-fd701f6916d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:41.556694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9300e4-8f81-4524-a859-fd701f6916d9 admin admin] PATH_INFO: `/v3/users/d24fd81503954a6e94d2eec0062882ed` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:41.557816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9300e4-8f81-4524-a859-fd701f6916d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d24fd81503954a6e94d2eec0062882ed)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:41.560135 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9300e4-8f81-4524-a859-fd701f6916d9 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:41.744525 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-8d9300e4-8f81-4524-a859-fd701f6916d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd24fd81503954a6e94d2eec0062882ed'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:41.838768 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-8d9300e4-8f81-4524-a859-fd701f6916d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd24fd81503954a6e94d2eec0062882ed'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:41.839621 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-8d9300e4-8f81-4524-a859-fd701f6916d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd24fd81503954a6e94d2eec0062882ed'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:41.847306 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-8d9300e4-8f81-4524-a859-fd701f6916d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd24fd81503954a6e94d2eec0062882ed'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:41.855245 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:41.860169 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-8d9300e4-8f81-4524-a859-fd701f6916d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd24fd81503954a6e94d2eec0062882ed'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:41.865975 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1432/2862] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:03:41 2022] DELETE /identity/v3/users/d24fd81503954a6e94d2eec0062882ed => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:42.143325 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553242e-ff6e-4df2-b7af-92364ecb2b72 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:42.144871 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-9553242e-ff6e-4df2-b7af-92364ecb2b72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:42.160590 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553242e-ff6e-4df2-b7af-92364ecb2b72 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:42.222897 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974561d-182d-4a4b-a60e-fd1dbfd1c50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:42.223559 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974561d-182d-4a4b-a60e-fd1dbfd1c50e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:42.223794 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974561d-182d-4a4b-a60e-fd1dbfd1c50e admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:42.224061 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974561d-182d-4a4b-a60e-fd1dbfd1c50e admin admin] PATH_INFO: `/v3/projects/b8b7a3ba3c29499eb4459d4db10bf0f8` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:42.225190 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974561d-182d-4a4b-a60e-fd1dbfd1c50e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8b7a3ba3c29499eb4459d4db10bf0f8)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:42.227128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974561d-182d-4a4b-a60e-fd1dbfd1c50e admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:42.354749 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7974561d-182d-4a4b-a60e-fd1dbfd1c50e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8b7a3ba3c29499eb4459d4db10bf0f8', 'request_id': 'req-7974561d-182d-4a4b-a60e-fd1dbfd1c50e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:42.355839 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:42.361393 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7974561d-182d-4a4b-a60e-fd1dbfd1c50e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8b7a3ba3c29499eb4459d4db10bf0f8', 'request_id': 'req-7974561d-182d-4a4b-a60e-fd1dbfd1c50e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:42.367586 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7974561d-182d-4a4b-a60e-fd1dbfd1c50e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8b7a3ba3c29499eb4459d4db10bf0f8', 'request_id': 'req-7974561d-182d-4a4b-a60e-fd1dbfd1c50e'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:42.464364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7974561d-182d-4a4b-a60e-fd1dbfd1c50e admin admin] The token cache is being invalidate because project b8b7a3ba3c29499eb4459d4db10bf0f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81998) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:03:42.464827 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7974561d-182d-4a4b-a60e-fd1dbfd1c50e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:42.465575 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-7974561d-182d-4a4b-a60e-fd1dbfd1c50e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:42.467404 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1431/2863] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:03:42 2022] DELETE /identity/v3/projects/b8b7a3ba3c29499eb4459d4db10bf0f8 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:42.473159 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48247a1b-958d-44ae-9118-45f584113f92 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:42.473445 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48247a1b-958d-44ae-9118-45f584113f92 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:42.473720 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48247a1b-958d-44ae-9118-45f584113f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:42.493539 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-48247a1b-958d-44ae-9118-45f584113f92 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:42.494793 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-48247a1b-958d-44ae-9118-45f584113f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:42.555718 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1433/2864] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:42.564009 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146da2d4-6814-4f35-a37a-dc064de8be0e None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:42.565839 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146da2d4-6814-4f35-a37a-dc064de8be0e None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:42.565839 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146da2d4-6814-4f35-a37a-dc064de8be0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:42.584857 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-146da2d4-6814-4f35-a37a-dc064de8be0e None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:42.586058 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-146da2d4-6814-4f35-a37a-dc064de8be0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:42.594960 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1432/2865] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:42.599325 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48247a1b-958d-44ae-9118-45f584113f92 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:42.609474 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48247a1b-958d-44ae-9118-45f584113f92 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:42.614350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6636e-8bb6-453e-8ff3-20697e9aad1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:42.615004 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6636e-8bb6-453e-8ff3-20697e9aad1c admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:42.615279 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6636e-8bb6-453e-8ff3-20697e9aad1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:42.615549 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6636e-8bb6-453e-8ff3-20697e9aad1c admin admin] PATH_INFO: `/v3/domains` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:42.616194 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6636e-8bb6-453e-8ff3-20697e9aad1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81999) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:03:42.616493 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6636e-8bb6-453e-8ff3-20697e9aad1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:42.618124 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6636e-8bb6-453e-8ff3-20697e9aad1c admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:42.630080 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1434/2866] 10.222.0.13 () {58 vars in 1153 bytes} [Mon Aug 8 17:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:03:42.634398 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146da2d4-6814-4f35-a37a-dc064de8be0e None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:42.648622 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146da2d4-6814-4f35-a37a-dc064de8be0e None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:42.653274 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce7975-8a3c-4d3d-bd23-11c36d8c7920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:42.653908 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce7975-8a3c-4d3d-bd23-11c36d8c7920 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:42.654165 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce7975-8a3c-4d3d-bd23-11c36d8c7920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:42.654460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce7975-8a3c-4d3d-bd23-11c36d8c7920 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:42.655718 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ce7975-8a3c-4d3d-bd23-11c36d8c7920 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:42.657685 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ce7975-8a3c-4d3d-bd23-11c36d8c7920 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:42.745892 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1433/2867] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:03:42 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:03:42.750678 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6636e-8bb6-453e-8ff3-20697e9aad1c admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:42.760588 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6636e-8bb6-453e-8ff3-20697e9aad1c admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:42.817528 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a563a8-8eca-4220-bc43-d780802cd156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:42.818139 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a563a8-8eca-4220-bc43-d780802cd156 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:42.818415 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a563a8-8eca-4220-bc43-d780802cd156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:42.818721 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a563a8-8eca-4220-bc43-d780802cd156 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:42.819339 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a563a8-8eca-4220-bc43-d780802cd156 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:42.821253 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a563a8-8eca-4220-bc43-d780802cd156 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:42.932703 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1435/2868] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:03:42 2022] POST /identity/v3/users => generated 290 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:03:42.937868 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce7975-8a3c-4d3d-bd23-11c36d8c7920 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:42.948630 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce7975-8a3c-4d3d-bd23-11c36d8c7920 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:42.954585 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e25044-9bdb-4f50-8811-b1293fe647f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:42.955195 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e25044-9bdb-4f50-8811-b1293fe647f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:42.955470 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e25044-9bdb-4f50-8811-b1293fe647f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:42.955735 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e25044-9bdb-4f50-8811-b1293fe647f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:42.956365 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e25044-9bdb-4f50-8811-b1293fe647f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:42.958119 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e25044-9bdb-4f50-8811-b1293fe647f0 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:42.968455 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1434/2869] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:03:42.972483 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a563a8-8eca-4220-bc43-d780802cd156 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:42.982969 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a563a8-8eca-4220-bc43-d780802cd156 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:42.987759 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea7f56-85b5-492f-a26c-fbf778bcef2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:42.988353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea7f56-85b5-492f-a26c-fbf778bcef2e admin admin] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:42.988632 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea7f56-85b5-492f-a26c-fbf778bcef2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:42.988904 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea7f56-85b5-492f-a26c-fbf778bcef2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:42.989540 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea7f56-85b5-492f-a26c-fbf778bcef2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:42.991166 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea7f56-85b5-492f-a26c-fbf778bcef2e admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:43.002463 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1436/2870] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:03:43.006625 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e25044-9bdb-4f50-8811-b1293fe647f0 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:43.017451 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e25044-9bdb-4f50-8811-b1293fe647f0 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:43.022128 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac430baa-98ca-4484-892a-6ada4320add6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:43.022806 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac430baa-98ca-4484-892a-6ada4320add6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:43.023079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac430baa-98ca-4484-892a-6ada4320add6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:43.023345 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac430baa-98ca-4484-892a-6ada4320add6 admin admin] PATH_INFO: `/v3/projects/f30f18b2d4fe4a9fa54fce853aa4c5cf/users/bd48464ff95749cd8b36b2fea90ddb31/roles/04bf40fcc26347de8167fc215f62c7aa` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:43.039022 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac430baa-98ca-4484-892a-6ada4320add6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f30f18b2d4fe4a9fa54fce853aa4c5cf, user_id=bd48464ff95749cd8b36b2fea90ddb31, role_id=04bf40fcc26347de8167fc215f62c7aa)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:43.041650 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac430baa-98ca-4484-892a-6ada4320add6 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:43.141439 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1435/2871] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:03:43 2022] PUT /identity/v3/projects/f30f18b2d4fe4a9fa54fce853aa4c5cf/users/bd48464ff95749cd8b36b2fea90ddb31/roles/04bf40fcc26347de8167fc215f62c7aa => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:43.146743 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe909ed-40e8-43c9-b0ca-af5979963390 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:43.147052 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe909ed-40e8-43c9-b0ca-af5979963390 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:43.147324 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe909ed-40e8-43c9-b0ca-af5979963390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:43.166481 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-5fe909ed-40e8-43c9-b0ca-af5979963390 None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:43.167704 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-5fe909ed-40e8-43c9-b0ca-af5979963390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:43.219598 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:43.229021 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1437/2872] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:43.239457 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac430baa-98ca-4484-892a-6ada4320add6 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:43.241193 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ac430baa-98ca-4484-892a-6ada4320add6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:43.302367 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac430baa-98ca-4484-892a-6ada4320add6 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:43.305947 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571021a-f401-4fca-95c0-4a775c80f977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:43.306546 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571021a-f401-4fca-95c0-4a775c80f977 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:43.306844 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571021a-f401-4fca-95c0-4a775c80f977 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:43.307106 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571021a-f401-4fca-95c0-4a775c80f977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:43.317079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571021a-f401-4fca-95c0-4a775c80f977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:43.318493 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571021a-f401-4fca-95c0-4a775c80f977 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:43.327925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1436/2873] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:03:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:03:48.911660 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4d211-3e9f-4680-861e-56e8aa9f65c8 None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:48.912203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4d211-3e9f-4680-861e-56e8aa9f65c8 None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:48.912203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4d211-3e9f-4680-861e-56e8aa9f65c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:48.942704 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-dae4d211-3e9f-4680-861e-56e8aa9f65c8 None None] MFA Rules not processed for user `bd48464ff95749cd8b36b2fea90ddb31`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:48.944202 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-dae4d211-3e9f-4680-861e-56e8aa9f65c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:49.012853 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1438/2874] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:03:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:49.018077 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571021a-f401-4fca-95c0-4a775c80f977 service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:49.027922 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571021a-f401-4fca-95c0-4a775c80f977 service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:49.033294 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116315d-5f80-4be3-b8d8-232d9ddb12b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:49.033889 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116315d-5f80-4be3-b8d8-232d9ddb12b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:49.034167 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116315d-5f80-4be3-b8d8-232d9ddb12b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:49.034460 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116315d-5f80-4be3-b8d8-232d9ddb12b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:49.035751 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116315d-5f80-4be3-b8d8-232d9ddb12b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:49.037193 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116315d-5f80-4be3-b8d8-232d9ddb12b9 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:49.134628 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1437/2875] 10.222.0.13 () {60 vars in 1153 bytes} [Mon Aug 8 17:03:49 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:03:49.138702 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4d211-3e9f-4680-861e-56e8aa9f65c8 None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:49.150583 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4d211-3e9f-4680-861e-56e8aa9f65c8 None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:49.215098 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f4838-3f31-4fd2-8d51-deb149483d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:49.215733 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f4838-3f31-4fd2-8d51-deb149483d02 admin admin] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:49.216018 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f4838-3f31-4fd2-8d51-deb149483d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:49.216289 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f4838-3f31-4fd2-8d51-deb149483d02 admin admin] PATH_INFO: `/v3/users` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:49.217015 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f4838-3f31-4fd2-8d51-deb149483d02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:49.219714 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f4838-3f31-4fd2-8d51-deb149483d02 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:49.333633 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1439/2876] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Aug 8 17:03:49 2022] POST /identity/v3/users => generated 295 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:03:49.338515 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116315d-5f80-4be3-b8d8-232d9ddb12b9 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:49.350755 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116315d-5f80-4be3-b8d8-232d9ddb12b9 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:49.355675 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db393b62-8b66-4257-a94d-aa6fea96a8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:49.356488 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db393b62-8b66-4257-a94d-aa6fea96a8be admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:49.356766 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db393b62-8b66-4257-a94d-aa6fea96a8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:49.357033 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db393b62-8b66-4257-a94d-aa6fea96a8be admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:49.357671 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db393b62-8b66-4257-a94d-aa6fea96a8be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:49.359907 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db393b62-8b66-4257-a94d-aa6fea96a8be admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:49.371124 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1438/2877] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:03:49.376952 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f4838-3f31-4fd2-8d51-deb149483d02 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:49.387443 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f4838-3f31-4fd2-8d51-deb149483d02 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:49.392478 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634760d0-643a-4c1e-82c5-2d00daa43e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:49.393156 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634760d0-643a-4c1e-82c5-2d00daa43e79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:49.393434 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634760d0-643a-4c1e-82c5-2d00daa43e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:49.393742 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634760d0-643a-4c1e-82c5-2d00daa43e79 admin admin] PATH_INFO: `/v3/projects/2aa05eb5c4ad487ab49ea58ba4ebd97d/users/28ac9c9809ca4b3e94109ab60c8e46f5/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:49.410989 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634760d0-643a-4c1e-82c5-2d00daa43e79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aa05eb5c4ad487ab49ea58ba4ebd97d, user_id=28ac9c9809ca4b3e94109ab60c8e46f5, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:49.413670 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634760d0-643a-4c1e-82c5-2d00daa43e79 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:49.530832 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1440/2878] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:03:49 2022] PUT /identity/v3/projects/2aa05eb5c4ad487ab49ea58ba4ebd97d/users/28ac9c9809ca4b3e94109ab60c8e46f5/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:49.531518 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db393b62-8b66-4257-a94d-aa6fea96a8be admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:49.540636 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db393b62-8b66-4257-a94d-aa6fea96a8be admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:49.588962 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:49.603617 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95bb42-51d8-495b-a52e-360d17e24a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:49.604688 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95bb42-51d8-495b-a52e-360d17e24a41 admin admin] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:49.604688 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95bb42-51d8-495b-a52e-360d17e24a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:49.604925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95bb42-51d8-495b-a52e-360d17e24a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:49.605457 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95bb42-51d8-495b-a52e-360d17e24a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:49.607095 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95bb42-51d8-495b-a52e-360d17e24a41 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:49.617440 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1439/2879] 10.222.0.13 () {58 vars in 1122 bytes} [Mon Aug 8 17:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:03:49.621639 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634760d0-643a-4c1e-82c5-2d00daa43e79 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:49.633982 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634760d0-643a-4c1e-82c5-2d00daa43e79 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:49.638745 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb6773-7b85-482c-b824-0982443e49d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:49.640015 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb6773-7b85-482c-b824-0982443e49d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:49.640694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb6773-7b85-482c-b824-0982443e49d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:49.640694 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb6773-7b85-482c-b824-0982443e49d9 admin admin] PATH_INFO: `/v3/projects/2aa05eb5c4ad487ab49ea58ba4ebd97d/users/28ac9c9809ca4b3e94109ab60c8e46f5/roles/e578ef831127463d9b48cc89fe20329c` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:49.642453 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb6773-7b85-482c-b824-0982443e49d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aa05eb5c4ad487ab49ea58ba4ebd97d, user_id=28ac9c9809ca4b3e94109ab60c8e46f5, role_id=e578ef831127463d9b48cc89fe20329c)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:49.645125 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb6773-7b85-482c-b824-0982443e49d9 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:49.656380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1441/2880] 10.222.0.13 () {60 vars in 1483 bytes} [Mon Aug 8 17:03:49 2022] PUT /identity/v3/projects/2aa05eb5c4ad487ab49ea58ba4ebd97d/users/28ac9c9809ca4b3e94109ab60c8e46f5/roles/e578ef831127463d9b48cc89fe20329c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:51.623083 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95bb42-51d8-495b-a52e-360d17e24a41 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:51.625064 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ef95bb42-51d8-495b-a52e-360d17e24a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:51.638673 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95bb42-51d8-495b-a52e-360d17e24a41 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:51.719067 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b19598-0f44-4a1f-b98e-bd9ea52740a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:51.720643 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b19598-0f44-4a1f-b98e-bd9ea52740a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:51.721080 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b19598-0f44-4a1f-b98e-bd9ea52740a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:51.721363 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b19598-0f44-4a1f-b98e-bd9ea52740a0 admin admin] PATH_INFO: `/v3/users/c94ac68dd4b241e9a7119e11be6a28f9` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:51.722644 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b19598-0f44-4a1f-b98e-bd9ea52740a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c94ac68dd4b241e9a7119e11be6a28f9)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:51.724657 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b19598-0f44-4a1f-b98e-bd9ea52740a0 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:52.026980 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-d1b19598-0f44-4a1f-b98e-bd9ea52740a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c94ac68dd4b241e9a7119e11be6a28f9'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:52.235823 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-d1b19598-0f44-4a1f-b98e-bd9ea52740a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c94ac68dd4b241e9a7119e11be6a28f9'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:52.236596 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-d1b19598-0f44-4a1f-b98e-bd9ea52740a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c94ac68dd4b241e9a7119e11be6a28f9'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:52.248097 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-d1b19598-0f44-4a1f-b98e-bd9ea52740a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c94ac68dd4b241e9a7119e11be6a28f9'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:52.251356 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-d1b19598-0f44-4a1f-b98e-bd9ea52740a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c94ac68dd4b241e9a7119e11be6a28f9'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:52.305104 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1440/2881] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:03:51 2022] DELETE /identity/v3/users/c94ac68dd4b241e9a7119e11be6a28f9 => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:52.311729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21116624-4f95-4498-825f-a5d0653be55f None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:52.311729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21116624-4f95-4498-825f-a5d0653be55f None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:52.311729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21116624-4f95-4498-825f-a5d0653be55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:52.337466 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-21116624-4f95-4498-825f-a5d0653be55f None None] MFA Rules not processed for user `b308731f39014d7098d27d547158b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:52.338981 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-21116624-4f95-4498-825f-a5d0653be55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:52.415046 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:52.449741 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1442/2882] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:52.464747 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b19598-0f44-4a1f-b98e-bd9ea52740a0 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:52.466458 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-d1b19598-0f44-4a1f-b98e-bd9ea52740a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:52.547819 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b19598-0f44-4a1f-b98e-bd9ea52740a0 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:52.551471 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07306b4-f07b-4aa6-89a6-9549975f52e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:52.552079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07306b4-f07b-4aa6-89a6-9549975f52e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:52.552510 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07306b4-f07b-4aa6-89a6-9549975f52e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:52.552785 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07306b4-f07b-4aa6-89a6-9549975f52e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:52.563269 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07306b4-f07b-4aa6-89a6-9549975f52e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:52.564745 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07306b4-f07b-4aa6-89a6-9549975f52e4 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:52.576323 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1441/2883] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:03:53.218278 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21116624-4f95-4498-825f-a5d0653be55f None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:53.218278 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-21116624-4f95-4498-825f-a5d0653be55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:53.233063 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21116624-4f95-4498-825f-a5d0653be55f None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:53.239067 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3833b-6112-464e-9496-97f2b2ea6d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:53.240158 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3833b-6112-464e-9496-97f2b2ea6d34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:53.240814 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3833b-6112-464e-9496-97f2b2ea6d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:53.241437 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3833b-6112-464e-9496-97f2b2ea6d34 admin admin] PATH_INFO: `/v3/projects/4ec5588e52df4d609e39fec525138f01` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:53.242873 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3833b-6112-464e-9496-97f2b2ea6d34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ec5588e52df4d609e39fec525138f01)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:53.245003 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3833b-6112-464e-9496-97f2b2ea6d34 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:53.387620 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b9a3833b-6112-464e-9496-97f2b2ea6d34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ec5588e52df4d609e39fec525138f01', 'request_id': 'req-b9a3833b-6112-464e-9496-97f2b2ea6d34'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:53.459297 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b9a3833b-6112-464e-9496-97f2b2ea6d34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ec5588e52df4d609e39fec525138f01', 'request_id': 'req-b9a3833b-6112-464e-9496-97f2b2ea6d34'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:53.465675 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b9a3833b-6112-464e-9496-97f2b2ea6d34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ec5588e52df4d609e39fec525138f01', 'request_id': 'req-b9a3833b-6112-464e-9496-97f2b2ea6d34'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:53.485486 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b9a3833b-6112-464e-9496-97f2b2ea6d34 admin admin] The token cache is being invalidate because project 4ec5588e52df4d609e39fec525138f01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:03:53.486520 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b9a3833b-6112-464e-9496-97f2b2ea6d34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:53.487637 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-b9a3833b-6112-464e-9496-97f2b2ea6d34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:53.495771 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1443/2884] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:03:53 2022] DELETE /identity/v3/projects/4ec5588e52df4d609e39fec525138f01 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:55.564242 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f8ca1-0c8e-4003-ad3a-b34189ba6422 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:55.564779 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f8ca1-0c8e-4003-ad3a-b34189ba6422 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:55.564908 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f8ca1-0c8e-4003-ad3a-b34189ba6422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:55.572115 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:55.601828 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-164f8ca1-0c8e-4003-ad3a-b34189ba6422 None None] MFA Rules not processed for user `28ac9c9809ca4b3e94109ab60c8e46f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:55.603393 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-164f8ca1-0c8e-4003-ad3a-b34189ba6422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:55.679729 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1442/2885] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:03:55 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:55.695658 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3833b-6112-464e-9496-97f2b2ea6d34 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:55.697124 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-b9a3833b-6112-464e-9496-97f2b2ea6d34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:55.782961 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3833b-6112-464e-9496-97f2b2ea6d34 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:55.788773 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30258d35-7242-4e50-be1e-772c0077518b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:55.789405 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30258d35-7242-4e50-be1e-772c0077518b service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:55.833336 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30258d35-7242-4e50-be1e-772c0077518b service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:55.833336 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30258d35-7242-4e50-be1e-772c0077518b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:55.833336 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-30258d35-7242-4e50-be1e-772c0077518b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:55.853706 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30258d35-7242-4e50-be1e-772c0077518b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:55.855288 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30258d35-7242-4e50-be1e-772c0077518b service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:55.860516 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:55.865119 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1444/2886] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:03:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:04:05.676866 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=33 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:04:14.218965 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20206844-6db3-4098-8553-c7f3aa6049dd None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:04:14.219684 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20206844-6db3-4098-8553-c7f3aa6049dd None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:04:14.219684 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20206844-6db3-4098-8553-c7f3aa6049dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:04:14.239624 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-20206844-6db3-4098-8553-c7f3aa6049dd None None] MFA Rules not processed for user `bd48464ff95749cd8b36b2fea90ddb31`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:04:14.240845 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-20206844-6db3-4098-8553-c7f3aa6049dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:04:14.248410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1443/2887] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:04:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:04:14.587151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30258d35-7242-4e50-be1e-772c0077518b service nova] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:04:14.588579 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-30258d35-7242-4e50-be1e-772c0077518b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:04:14.661036 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30258d35-7242-4e50-be1e-772c0077518b service nova] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:04:14.664979 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb9d4c-2ef6-4c2a-b6cb-e602a26165c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:04:14.665601 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb9d4c-2ef6-4c2a-b6cb-e602a26165c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:04:14.665878 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb9d4c-2ef6-4c2a-b6cb-e602a26165c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:04:14.666151 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb9d4c-2ef6-4c2a-b6cb-e602a26165c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:04:14.668034 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-5eeb9d4c-2ef6-4c2a-b6cb-e602a26165c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:04:14.679194 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb9d4c-2ef6-4c2a-b6cb-e602a26165c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:04:14.680648 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb9d4c-2ef6-4c2a-b6cb-e602a26165c7 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:04:14.691762 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1445/2888] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:04:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:04:15.629520 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20206844-6db3-4098-8553-c7f3aa6049dd None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:04:15.630986 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-20206844-6db3-4098-8553-c7f3aa6049dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:04:15.712039 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20206844-6db3-4098-8553-c7f3aa6049dd None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:04:15.717984 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d19ff-823a-450b-b38d-9d59bad2876c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e74dbb121d749749cf5f9d9d7221008', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:04:15.718649 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d19ff-823a-450b-b38d-9d59bad2876c service placement] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:04:15.718927 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d19ff-823a-450b-b38d-9d59bad2876c service placement] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:04:15.719194 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d19ff-823a-450b-b38d-9d59bad2876c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:04:15.721114 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-5a9d19ff-823a-450b-b38d-9d59bad2876c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:04:15.734813 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d19ff-823a-450b-b38d-9d59bad2876c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:04:15.736454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d19ff-823a-450b-b38d-9d59bad2876c service placement] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:04:15.747454 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1444/2889] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:04:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:04:24.685880 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=26 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:04:25.742911 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=35 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:04:30.997907 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb9d4c-2ef6-4c2a-b6cb-e602a26165c7 service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:04:30.999942 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-5eeb9d4c-2ef6-4c2a-b6cb-e602a26165c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:04:31.017049 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb9d4c-2ef6-4c2a-b6cb-e602a26165c7 service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:04:31.022816 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814a700-34c8-4b0a-8688-32bfcf64b293 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:04:31.029674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814a700-34c8-4b0a-8688-32bfcf64b293 service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:04:31.029674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814a700-34c8-4b0a-8688-32bfcf64b293 service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:04:31.029674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814a700-34c8-4b0a-8688-32bfcf64b293 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:04:31.029674 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-4814a700-34c8-4b0a-8688-32bfcf64b293 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:04:31.038501 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814a700-34c8-4b0a-8688-32bfcf64b293 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:04:31.040247 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814a700-34c8-4b0a-8688-32bfcf64b293 service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:04:31.051891 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1446/2890] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:04:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:04:33.417398 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d19ff-823a-450b-b38d-9d59bad2876c service placement] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:04:33.420127 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-5a9d19ff-823a-450b-b38d-9d59bad2876c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:04:33.431432 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d19ff-823a-450b-b38d-9d59bad2876c service placement] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:04:33.436379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55d913-9ec8-4523-aa3f-5996c04696a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:04:33.436961 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55d913-9ec8-4523-aa3f-5996c04696a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:04:33.437235 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55d913-9ec8-4523-aa3f-5996c04696a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:04:33.437498 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55d913-9ec8-4523-aa3f-5996c04696a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:04:33.446456 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c55d913-9ec8-4523-aa3f-5996c04696a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:04:33.447938 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c55d913-9ec8-4523-aa3f-5996c04696a9 service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:04:33.458283 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1445/2891] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:04:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:04:41.047276 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=6 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:04:43.453881 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=6 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:05:06.871237 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814a700-34c8-4b0a-8688-32bfcf64b293 service nova] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:05:06.884511 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814a700-34c8-4b0a-8688-32bfcf64b293 service nova] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:05:06.890229 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a09db-ffbe-484d-bddb-b056cb805462 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e74dbb121d749749cf5f9d9d7221008', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:05:06.891011 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a09db-ffbe-484d-bddb-b056cb805462 service placement] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:05:06.891293 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a09db-ffbe-484d-bddb-b056cb805462 service placement] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:05:06.891560 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a09db-ffbe-484d-bddb-b056cb805462 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:05:06.893406 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-8a1a09db-ffbe-484d-bddb-b056cb805462 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:05:06.906865 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a09db-ffbe-484d-bddb-b056cb805462 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:05:06.908498 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a09db-ffbe-484d-bddb-b056cb805462 service placement] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:05:06.919872 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1447/2892] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:05:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:05:16.914961 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=6 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:06:35.649784 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dad91a-9b79-47b5-ac57-ea480aa59f15 None None] REQUEST_METHOD: `POST` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:06:35.650309 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dad91a-9b79-47b5-ac57-ea480aa59f15 None None] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:06:35.650309 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dad91a-9b79-47b5-ac57-ea480aa59f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:06:35.669234 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.auth.core [None req-a6dad91a-9b79-47b5-ac57-ea480aa59f15 None None] MFA Rules not processed for user `bd48464ff95749cd8b36b2fea90ddb31`. Rule list: `[]` (Enabled: `True`). {{(pid=81998) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:06:35.670430 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-a6dad91a-9b79-47b5-ac57-ea480aa59f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:06:35.677245 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1446/2893] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:06:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:06:45.664352 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=3 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:06:47.096062 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a09db-ffbe-484d-bddb-b056cb805462 service placement] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:06:47.106548 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a09db-ffbe-484d-bddb-b056cb805462 service placement] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:06:47.110202 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0daeaa-b459-4e9c-8223-41e6fb1e708d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:06:47.110842 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0daeaa-b459-4e9c-8223-41e6fb1e708d service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:06:47.111117 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0daeaa-b459-4e9c-8223-41e6fb1e708d service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:06:47.111384 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0daeaa-b459-4e9c-8223-41e6fb1e708d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:06:47.113090 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-4e0daeaa-b459-4e9c-8223-41e6fb1e708d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:06:47.127404 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0daeaa-b459-4e9c-8223-41e6fb1e708d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:06:47.128807 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0daeaa-b459-4e9c-8223-41e6fb1e708d service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:06:47.140592 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1448/2894] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:06:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:06:48.332063 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dad91a-9b79-47b5-ac57-ea480aa59f15 None None] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:06:48.344531 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dad91a-9b79-47b5-ac57-ea480aa59f15 None None] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:06:48.348526 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84debcec-8e5c-47f3-afbe-85dc51d3610a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:06:48.349129 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84debcec-8e5c-47f3-afbe-85dc51d3610a service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:06:48.349413 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84debcec-8e5c-47f3-afbe-85dc51d3610a service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:06:48.349679 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84debcec-8e5c-47f3-afbe-85dc51d3610a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:06:48.351429 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-84debcec-8e5c-47f3-afbe-85dc51d3610a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:06:48.365760 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84debcec-8e5c-47f3-afbe-85dc51d3610a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:06:48.367249 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84debcec-8e5c-47f3-afbe-85dc51d3610a service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:06:48.379773 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1447/2895] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:06:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:06:57.135098 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=6 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:06:58.374497 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=6 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:08:34.973561 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e0923-1b5b-488f-bcc8-4184b47efcdb None None] REQUEST_METHOD: `POST` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:08:34.978758 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e0923-1b5b-488f-bcc8-4184b47efcdb None None] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:08:34.978758 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e0923-1b5b-488f-bcc8-4184b47efcdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:08:34.993813 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.auth.core [None req-aa4e0923-1b5b-488f-bcc8-4184b47efcdb None None] MFA Rules not processed for user `bd48464ff95749cd8b36b2fea90ddb31`. Rule list: `[]` (Enabled: `True`). {{(pid=81999) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:08:34.995055 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-aa4e0923-1b5b-488f-bcc8-4184b47efcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:08:35.001887 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1449/2896] 10.222.0.13 () {58 vars in 958 bytes} [Mon Aug 8 17:08:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:08:38.976309 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84debcec-8e5c-47f3-afbe-85dc51d3610a service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:08:38.985421 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84debcec-8e5c-47f3-afbe-85dc51d3610a service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:08:38.990001 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cbab61-a31a-4c32-b4c6-11946667fbba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:08:38.990925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbab61-a31a-4c32-b4c6-11946667fbba service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:08:38.990925 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbab61-a31a-4c32-b4c6-11946667fbba service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:08:38.991171 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbab61-a31a-4c32-b4c6-11946667fbba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:08:38.992944 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-75cbab61-a31a-4c32-b4c6-11946667fbba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:08:39.004723 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cbab61-a31a-4c32-b4c6-11946667fbba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:08:39.005686 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cbab61-a31a-4c32-b4c6-11946667fbba service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:08:39.014270 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1448/2897] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:08:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:08:39.855796 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e0923-1b5b-488f-bcc8-4184b47efcdb None None] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:08:39.863969 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e0923-1b5b-488f-bcc8-4184b47efcdb None None] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:08:39.871288 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87063d2-603a-4584-9e21-dd8328acda25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daa7489bced467984d39e8b8ac49d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:08:39.871920 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87063d2-603a-4584-9e21-dd8328acda25 service nova] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:08:39.872203 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87063d2-603a-4584-9e21-dd8328acda25 service nova] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:08:39.872523 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87063d2-603a-4584-9e21-dd8328acda25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:08:39.874380 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-c87063d2-603a-4584-9e21-dd8328acda25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:08:39.887397 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87063d2-603a-4584-9e21-dd8328acda25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:08:39.901625 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87063d2-603a-4584-9e21-dd8328acda25 service nova] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:08:39.901625 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1450/2898] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:08:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:08:49.010410 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=6 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:08:49.896134 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=9 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:10:08.833993 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cbab61-a31a-4c32-b4c6-11946667fbba service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:10:08.907779 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cbab61-a31a-4c32-b4c6-11946667fbba service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:10:08.913218 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976af379-6652-48e4-99ad-c3791f635652 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e74dbb121d749749cf5f9d9d7221008', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:10:08.913842 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976af379-6652-48e4-99ad-c3791f635652 service placement] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:10:08.914117 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976af379-6652-48e4-99ad-c3791f635652 service placement] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:10:08.914386 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976af379-6652-48e4-99ad-c3791f635652 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:10:08.926118 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976af379-6652-48e4-99ad-c3791f635652 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:10:08.928250 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976af379-6652-48e4-99ad-c3791f635652 service placement] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:10:08.940378 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1449/2899] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:10:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:10:17.289729 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87063d2-603a-4584-9e21-dd8328acda25 service nova] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:10:17.330080 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87063d2-603a-4584-9e21-dd8328acda25 service nova] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:10:17.333738 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5604dc-f247-4892-bb65-947e538b52d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:10:17.334339 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5604dc-f247-4892-bb65-947e538b52d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:10:17.334640 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5604dc-f247-4892-bb65-947e538b52d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:10:17.334908 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5604dc-f247-4892-bb65-947e538b52d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:10:17.343356 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5604dc-f247-4892-bb65-947e538b52d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:10:17.344767 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5604dc-f247-4892-bb65-947e538b52d4 service neutron] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:10:17.354045 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1451/2900] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:10:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:10:18.935379 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=27 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:10:27.349502 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=18 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:10:31.895906 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976af379-6652-48e4-99ad-c3791f635652 service placement] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:10:31.905905 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976af379-6652-48e4-99ad-c3791f635652 service placement] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:10:31.914438 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad388471-c16f-4a2b-8b32-a4dbeffdeffb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9482bf89c1b648c1a91ee00ee5fbba63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:10:31.915069 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad388471-c16f-4a2b-8b32-a4dbeffdeffb service neutron] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:10:31.915352 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad388471-c16f-4a2b-8b32-a4dbeffdeffb service neutron] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:10:31.915648 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad388471-c16f-4a2b-8b32-a4dbeffdeffb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:10:31.917364 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ad388471-c16f-4a2b-8b32-a4dbeffdeffb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:10:32.015697 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad388471-c16f-4a2b-8b32-a4dbeffdeffb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:10:32.017142 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad388471-c16f-4a2b-8b32-a4dbeffdeffb service neutron] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:10:32.036323 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1450/2901] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:10:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:10:35.578782 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5604dc-f247-4892-bb65-947e538b52d4 service neutron] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:10:35.580238 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-ee5604dc-f247-4892-bb65-947e538b52d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:10:35.609276 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5604dc-f247-4892-bb65-947e538b52d4 service neutron] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:10:35.614309 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5ab3b-cf5c-4578-9171-e48f6a089bee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e74dbb121d749749cf5f9d9d7221008', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:10:35.614958 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5ab3b-cf5c-4578-9171-e48f6a089bee service placement] REQUEST_METHOD: `GET` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:10:35.620699 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5ab3b-cf5c-4578-9171-e48f6a089bee service placement] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:10:35.620699 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5ab3b-cf5c-4578-9171-e48f6a089bee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:10:35.620699 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-02e5ab3b-cf5c-4578-9171-e48f6a089bee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:10:35.629423 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5ab3b-cf5c-4578-9171-e48f6a089bee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:10:35.631091 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5ab3b-cf5c-4578-9171-e48f6a089bee service placement] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:10:35.641520 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1452/2902] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:10:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:10:39.455398 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad388471-c16f-4a2b-8b32-a4dbeffdeffb service neutron] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:10:39.456964 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-ad388471-c16f-4a2b-8b32-a4dbeffdeffb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:10:39.470164 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad388471-c16f-4a2b-8b32-a4dbeffdeffb service neutron] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:10:39.474803 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391e87c-4f6c-4d24-a977-0270bfcbe1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:10:39.475429 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391e87c-4f6c-4d24-a977-0270bfcbe1c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:10:39.475705 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391e87c-4f6c-4d24-a977-0270bfcbe1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:10:39.475996 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391e87c-4f6c-4d24-a977-0270bfcbe1c5 admin admin] PATH_INFO: `/v3/users/bd48464ff95749cd8b36b2fea90ddb31` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:10:39.477109 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391e87c-4f6c-4d24-a977-0270bfcbe1c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd48464ff95749cd8b36b2fea90ddb31)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:10:39.479074 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391e87c-4f6c-4d24-a977-0270bfcbe1c5 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:10:39.638457 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-9391e87c-4f6c-4d24-a977-0270bfcbe1c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd48464ff95749cd8b36b2fea90ddb31'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:10:39.693913 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-9391e87c-4f6c-4d24-a977-0270bfcbe1c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd48464ff95749cd8b36b2fea90ddb31'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:10:39.694525 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-9391e87c-4f6c-4d24-a977-0270bfcbe1c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd48464ff95749cd8b36b2fea90ddb31'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:10:39.702561 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-9391e87c-4f6c-4d24-a977-0270bfcbe1c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd48464ff95749cd8b36b2fea90ddb31'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:10:39.706522 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-9391e87c-4f6c-4d24-a977-0270bfcbe1c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd48464ff95749cd8b36b2fea90ddb31'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:10:39.721990 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1451/2903] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:10:39 2022] DELETE /identity/v3/users/bd48464ff95749cd8b36b2fea90ddb31 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:10:39.964179 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5ab3b-cf5c-4578-9171-e48f6a089bee service placement] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:10:39.977026 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5ab3b-cf5c-4578-9171-e48f6a089bee service placement] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:10:40.036341 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c45b46c-a7c1-4c23-b65c-a04dfe00b873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:10:40.037002 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45b46c-a7c1-4c23-b65c-a04dfe00b873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:10:40.037274 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45b46c-a7c1-4c23-b65c-a04dfe00b873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:10:40.037542 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45b46c-a7c1-4c23-b65c-a04dfe00b873 admin admin] PATH_INFO: `/v3/projects/f30f18b2d4fe4a9fa54fce853aa4c5cf` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:10:40.038666 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c45b46c-a7c1-4c23-b65c-a04dfe00b873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f30f18b2d4fe4a9fa54fce853aa4c5cf)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:10:40.040571 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c45b46c-a7c1-4c23-b65c-a04dfe00b873 admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:10:40.151994 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-7c45b46c-a7c1-4c23-b65c-a04dfe00b873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f30f18b2d4fe4a9fa54fce853aa4c5cf', 'request_id': 'req-7c45b46c-a7c1-4c23-b65c-a04dfe00b873'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:10:40.219350 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-7c45b46c-a7c1-4c23-b65c-a04dfe00b873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f30f18b2d4fe4a9fa54fce853aa4c5cf', 'request_id': 'req-7c45b46c-a7c1-4c23-b65c-a04dfe00b873'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:10:40.226353 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-7c45b46c-a7c1-4c23-b65c-a04dfe00b873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f30f18b2d4fe4a9fa54fce853aa4c5cf', 'request_id': 'req-7c45b46c-a7c1-4c23-b65c-a04dfe00b873'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:10:40.237104 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-7c45b46c-a7c1-4c23-b65c-a04dfe00b873 admin admin] The token cache is being invalidate because project f30f18b2d4fe4a9fa54fce853aa4c5cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:10:40.237387 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-7c45b46c-a7c1-4c23-b65c-a04dfe00b873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:10:40.238057 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-7c45b46c-a7c1-4c23-b65c-a04dfe00b873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:10:40.239579 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1453/2904] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:10:39 2022] DELETE /identity/v3/projects/f30f18b2d4fe4a9fa54fce853aa4c5cf => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:10:40.243756 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391e87c-4f6c-4d24-a977-0270bfcbe1c5 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:10:40.245177 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-9391e87c-4f6c-4d24-a977-0270bfcbe1c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:10:40.259624 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391e87c-4f6c-4d24-a977-0270bfcbe1c5 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:10:40.287695 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:10:40.321171 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566f8b8-739a-4fc3-9ac9-393cb85fba46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:10:40.321804 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566f8b8-739a-4fc3-9ac9-393cb85fba46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:10:40.322079 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566f8b8-739a-4fc3-9ac9-393cb85fba46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:10:40.322348 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566f8b8-739a-4fc3-9ac9-393cb85fba46 admin admin] PATH_INFO: `/v3/users/28ac9c9809ca4b3e94109ab60c8e46f5` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:10:40.323537 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566f8b8-739a-4fc3-9ac9-393cb85fba46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28ac9c9809ca4b3e94109ab60c8e46f5)` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:10:40.325514 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566f8b8-739a-4fc3-9ac9-393cb85fba46 admin admin] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:10:40.515383 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-0566f8b8-739a-4fc3-9ac9-393cb85fba46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28ac9c9809ca4b3e94109ab60c8e46f5'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:10:40.585317 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-0566f8b8-739a-4fc3-9ac9-393cb85fba46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28ac9c9809ca4b3e94109ab60c8e46f5'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:10:40.585912 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-0566f8b8-739a-4fc3-9ac9-393cb85fba46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28ac9c9809ca4b3e94109ab60c8e46f5'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:10:40.593473 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-0566f8b8-739a-4fc3-9ac9-393cb85fba46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28ac9c9809ca4b3e94109ab60c8e46f5'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:10:40.594102 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.notifications [None req-0566f8b8-739a-4fc3-9ac9-393cb85fba46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28ac9c9809ca4b3e94109ab60c8e46f5'} {{(pid=81998) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:10:40.611731 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1452/2905] 10.222.0.13 () {58 vars in 1224 bytes} [Mon Aug 8 17:10:40 2022] DELETE /identity/v3/users/28ac9c9809ca4b3e94109ab60c8e46f5 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:10:40.861234 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c45b46c-a7c1-4c23-b65c-a04dfe00b873 admin admin] Authenticating user token {{(pid=81999) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:10:40.864536 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.fernet_utils [None req-7c45b46c-a7c1-4c23-b65c-a04dfe00b873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81999) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:10:40.876282 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c45b46c-a7c1-4c23-b65c-a04dfe00b873 admin admin] Validating token access rules against request {{(pid=81999) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:10:40.934313 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc532c-09ea-475d-a64a-4508278ba6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b308731f39014d7098d27d547158b386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a8d1eb28476411285e5ac5c0ab9ab53', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81999) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:10:40.935020 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc532c-09ea-475d-a64a-4508278ba6ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:10:40.935287 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc532c-09ea-475d-a64a-4508278ba6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:10:40.935555 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc532c-09ea-475d-a64a-4508278ba6ef admin admin] PATH_INFO: `/v3/projects/2aa05eb5c4ad487ab49ea58ba4ebd97d` {{(pid=81999) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:10:40.937434 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc532c-09ea-475d-a64a-4508278ba6ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aa05eb5c4ad487ab49ea58ba4ebd97d)` {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:10:40.939373 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc532c-09ea-475d-a64a-4508278ba6ef admin admin] RBAC: Authorization granted {{(pid=81999) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:10:40.947702 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:10:41.048810 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-cfbc532c-09ea-475d-a64a-4508278ba6ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aa05eb5c4ad487ab49ea58ba4ebd97d', 'request_id': 'req-cfbc532c-09ea-475d-a64a-4508278ba6ef'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:10:41.150969 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-cfbc532c-09ea-475d-a64a-4508278ba6ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aa05eb5c4ad487ab49ea58ba4ebd97d', 'request_id': 'req-cfbc532c-09ea-475d-a64a-4508278ba6ef'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:10:41.158861 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-cfbc532c-09ea-475d-a64a-4508278ba6ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aa05eb5c4ad487ab49ea58ba4ebd97d', 'request_id': 'req-cfbc532c-09ea-475d-a64a-4508278ba6ef'} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:10:41.169482 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-cfbc532c-09ea-475d-a64a-4508278ba6ef admin admin] The token cache is being invalidate because project 2aa05eb5c4ad487ab49ea58ba4ebd97d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81999) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:10:41.169771 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-cfbc532c-09ea-475d-a64a-4508278ba6ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:10:41.170427 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG keystone.notifications [None req-cfbc532c-09ea-475d-a64a-4508278ba6ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81999) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:10:41.172684 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: [pid: 81999|app: 0|req: 1454/2906] 10.222.0.13 () {58 vars in 1233 bytes} [Mon Aug 8 17:10:40 2022] DELETE /identity/v3/projects/2aa05eb5c4ad487ab49ea58ba4ebd97d => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:10:50.607694 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=40,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:10:51.166469 dv-cinder-852209-3-iscsi devstack@keystone.service[81999]: DEBUG dbcounter [-] [81999] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81999) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:15:10.844944 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566f8b8-739a-4fc3-9ac9-393cb85fba46 admin admin] Authenticating user token {{(pid=81998) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:15:10.846342 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-0566f8b8-739a-4fc3-9ac9-393cb85fba46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:15:10.917743 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566f8b8-739a-4fc3-9ac9-393cb85fba46 admin admin] Validating token access rules against request {{(pid=81998) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:15:10.977671 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d0d1c-8d97-4abe-b882-2f8d4bf0ce91 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e74dbb121d749749cf5f9d9d7221008', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56ab2fc79d964f4d86edefb4d1b2c993', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81998) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:15:10.978275 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d0d1c-8d97-4abe-b882-2f8d4bf0ce91 service placement] REQUEST_METHOD: `GET` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:15:10.978549 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d0d1c-8d97-4abe-b882-2f8d4bf0ce91 service placement] SCRIPT_NAME: `/identity` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:15:10.978842 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d0d1c-8d97-4abe-b882-2f8d4bf0ce91 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81998) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:15:10.980598 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.fernet_utils [None req-4a7d0d1c-8d97-4abe-b882-2f8d4bf0ce91 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81998) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:15:10.992726 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d0d1c-8d97-4abe-b882-2f8d4bf0ce91 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:15:10.995513 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d0d1c-8d97-4abe-b882-2f8d4bf0ce91 service placement] RBAC: Authorization granted {{(pid=81998) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:15:11.004267 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: [pid: 81998|app: 0|req: 1453/2907] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Aug 8 17:15:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:15:20.999995 dv-cinder-852209-3-iscsi devstack@keystone.service[81998]: DEBUG dbcounter [-] [81998] Writing DB stats keystone:SELECT=58 {{(pid=81998) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}